2023-03-27 03:53:32,535 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-03-27 03:53:32,822 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7 2023-03-27 03:53:32,834 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-03-27 03:53:32,835 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/cluster_3244e29f-4978-4f89-598f-7715da9c7d7c, deleteOnExit=true 2023-03-27 03:53:32,835 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-03-27 03:53:32,835 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/test.cache.data in system properties and HBase conf 2023-03-27 03:53:32,836 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/hadoop.tmp.dir in system properties and HBase conf 2023-03-27 03:53:32,836 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/hadoop.log.dir in system properties and HBase conf 2023-03-27 03:53:32,837 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/mapreduce.cluster.local.dir in system properties and HBase conf 2023-03-27 03:53:32,837 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-03-27 03:53:32,837 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-03-27 03:53:32,952 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-03-27 03:53:33,373 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-03-27 03:53:33,378 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-03-27 03:53:33,378 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-03-27 03:53:33,379 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-03-27 03:53:33,379 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-27 03:53:33,379 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-03-27 03:53:33,380 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-03-27 03:53:33,380 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-03-27 03:53:33,380 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-27 03:53:33,381 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-03-27 03:53:33,381 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/nfs.dump.dir in system properties and HBase conf 2023-03-27 03:53:33,382 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/java.io.tmpdir in system properties and HBase conf 2023-03-27 03:53:33,382 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/dfs.journalnode.edits.dir in system properties and HBase conf 2023-03-27 03:53:33,382 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-03-27 03:53:33,383 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-03-27 03:53:33,945 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-27 03:53:33,950 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-27 03:53:34,257 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-03-27 03:53:34,443 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-03-27 03:53:34,461 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-27 03:53:34,502 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-03-27 03:53:34,535 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/java.io.tmpdir/Jetty_localhost_43489_hdfs____.7scr9c/webapp 2023-03-27 03:53:34,679 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43489 2023-03-27 03:53:34,715 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-03-27 03:53:34,716 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-03-27 03:53:35,151 WARN [Listener at localhost/43651] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-27 03:53:35,292 WARN [Listener at localhost/43651] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-03-27 03:53:35,312 WARN [Listener at localhost/43651] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-03-27 03:53:35,319 INFO [Listener at localhost/43651] log.Slf4jLog(67): jetty-6.1.26 2023-03-27 03:53:35,326 INFO [Listener at localhost/43651] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/java.io.tmpdir/Jetty_localhost_34079_datanode____.35g2h4/webapp 2023-03-27 03:53:35,462 INFO [Listener at localhost/43651] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34079 2023-03-27 03:53:35,896 WARN [Listener at localhost/36131] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-03-27 03:53:36,443 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xafa86d84756f9bbc: Processing first storage report for DS-0a811c54-aac2-43e0-90a5-2921a697cf04 from datanode 934c2615-be1e-4545-af16-f1d75719e819 2023-03-27 03:53:36,443 DEBUG [Listener at localhost/36131] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7 2023-03-27 03:53:36,445 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xafa86d84756f9bbc: from storage DS-0a811c54-aac2-43e0-90a5-2921a697cf04 node DatanodeRegistration(127.0.0.1:34783, datanodeUuid=934c2615-be1e-4545-af16-f1d75719e819, infoPort=35249, infoSecurePort=0, ipcPort=36131, storageInfo=lv=-57;cid=testClusterID;nsid=1447236593;c=1679889214022), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-03-27 03:53:36,445 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xafa86d84756f9bbc: Processing first storage report for DS-a49bd638-1dc3-49fa-bda7-4ad91467b9ac from datanode 934c2615-be1e-4545-af16-f1d75719e819 2023-03-27 03:53:36,445 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xafa86d84756f9bbc: from storage DS-a49bd638-1dc3-49fa-bda7-4ad91467b9ac node DatanodeRegistration(127.0.0.1:34783, datanodeUuid=934c2615-be1e-4545-af16-f1d75719e819, infoPort=35249, infoSecurePort=0, ipcPort=36131, storageInfo=lv=-57;cid=testClusterID;nsid=1447236593;c=1679889214022), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-03-27 03:53:36,503 INFO [Listener at localhost/36131] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/cluster_3244e29f-4978-4f89-598f-7715da9c7d7c/zookeeper_0, clientPort=62246, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/cluster_3244e29f-4978-4f89-598f-7715da9c7d7c/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/cluster_3244e29f-4978-4f89-598f-7715da9c7d7c/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-03-27 03:53:36,516 INFO [Listener at localhost/36131] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=62246 2023-03-27 03:53:36,526 INFO [Listener at localhost/36131] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-27 03:53:36,528 INFO [Listener at localhost/36131] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-27 03:53:37,204 INFO [Listener at localhost/36131] util.FSUtils(471): Created version file at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001 with version=8 2023-03-27 03:53:37,205 INFO [Listener at localhost/36131] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/hbase-staging 2023-03-27 03:53:37,513 INFO [Listener at localhost/36131] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-03-27 03:53:37,998 INFO [Listener at localhost/36131] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-03-27 03:53:38,033 INFO [Listener at localhost/36131] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-27 03:53:38,034 INFO [Listener at localhost/36131] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-27 03:53:38,034 INFO [Listener at localhost/36131] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-27 03:53:38,035 INFO [Listener at localhost/36131] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-27 03:53:38,035 INFO [Listener at localhost/36131] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-27 03:53:38,186 INFO [Listener at localhost/36131] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-03-27 03:53:38,270 DEBUG [Listener at localhost/36131] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-03-27 03:53:38,366 INFO [Listener at localhost/36131] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38271 2023-03-27 03:53:38,376 INFO [Listener at localhost/36131] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-27 03:53:38,378 INFO [Listener at localhost/36131] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-27 03:53:38,398 INFO [Listener at localhost/36131] zookeeper.RecoverableZooKeeper(93): Process identifier=master:38271 connecting to ZooKeeper ensemble=127.0.0.1:62246 2023-03-27 03:53:38,437 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:382710x0, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-27 03:53:38,440 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:38271-0x101241b1daf0000 connected 2023-03-27 03:53:38,465 DEBUG [Listener at localhost/36131] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-27 03:53:38,466 DEBUG [Listener at localhost/36131] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-27 03:53:38,469 DEBUG [Listener at localhost/36131] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-27 03:53:38,477 DEBUG [Listener at localhost/36131] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38271 2023-03-27 03:53:38,478 DEBUG [Listener at localhost/36131] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38271 2023-03-27 03:53:38,478 DEBUG [Listener at localhost/36131] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38271 2023-03-27 03:53:38,480 DEBUG [Listener at localhost/36131] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38271 2023-03-27 03:53:38,480 DEBUG [Listener at localhost/36131] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38271 2023-03-27 03:53:38,486 INFO [Listener at localhost/36131] master.HMaster(444): hbase.rootdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001, hbase.cluster.distributed=false 2023-03-27 03:53:38,561 INFO [Listener at localhost/36131] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-03-27 03:53:38,561 INFO [Listener at localhost/36131] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-27 03:53:38,561 INFO [Listener at localhost/36131] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-03-27 03:53:38,561 INFO [Listener at localhost/36131] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-03-27 03:53:38,561 INFO [Listener at localhost/36131] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-03-27 03:53:38,562 INFO [Listener at localhost/36131] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-03-27 03:53:38,568 INFO [Listener at localhost/36131] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-03-27 03:53:38,572 INFO [Listener at localhost/36131] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39727 2023-03-27 03:53:38,575 INFO [Listener at localhost/36131] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-03-27 03:53:38,583 DEBUG [Listener at localhost/36131] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-03-27 03:53:38,584 INFO [Listener at localhost/36131] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-27 03:53:38,588 INFO [Listener at localhost/36131] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-27 03:53:38,590 INFO [Listener at localhost/36131] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:39727 connecting to ZooKeeper ensemble=127.0.0.1:62246 2023-03-27 03:53:38,594 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:397270x0, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-03-27 03:53:38,595 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:39727-0x101241b1daf0001 connected 2023-03-27 03:53:38,595 DEBUG [Listener at localhost/36131] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-27 03:53:38,596 DEBUG [Listener at localhost/36131] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-27 03:53:38,597 DEBUG [Listener at localhost/36131] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-03-27 03:53:38,598 DEBUG [Listener at localhost/36131] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39727 2023-03-27 03:53:38,598 DEBUG [Listener at localhost/36131] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39727 2023-03-27 03:53:38,599 DEBUG [Listener at localhost/36131] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39727 2023-03-27 03:53:38,599 DEBUG [Listener at localhost/36131] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39727 2023-03-27 03:53:38,599 DEBUG [Listener at localhost/36131] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39727 2023-03-27 03:53:38,605 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,38271,1679889217353 2023-03-27 03:53:38,614 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-27 03:53:38,615 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,38271,1679889217353 2023-03-27 03:53:38,634 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-27 03:53:38,634 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-03-27 03:53:38,634 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-27 03:53:38,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-27 03:53:38,636 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,38271,1679889217353 from backup master directory 2023-03-27 03:53:38,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-03-27 03:53:38,639 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,38271,1679889217353 2023-03-27 03:53:38,639 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-03-27 03:53:38,640 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-27 03:53:38,640 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,38271,1679889217353 2023-03-27 03:53:38,643 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-03-27 03:53:38,644 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-03-27 03:53:38,736 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/hbase.id with ID: 54847249-82a7-4951-a85d-e1357ad5c94f 2023-03-27 03:53:38,780 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-27 03:53:38,794 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-27 03:53:38,835 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x5a340601 to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:53:38,872 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@79a78b30, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:53:38,899 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-03-27 03:53:38,901 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-03-27 03:53:38,925 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-03-27 03:53:38,925 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-03-27 03:53:38,927 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-27 03:53:38,932 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-03-27 03:53:38,933 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-27 03:53:38,966 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/MasterData/data/master/store-tmp 2023-03-27 03:53:39,396 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:53:39,396 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-27 03:53:39,397 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-27 03:53:39,397 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-27 03:53:39,397 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-27 03:53:39,397 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-27 03:53:39,397 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-27 03:53:39,397 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-27 03:53:39,399 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/MasterData/WALs/jenkins-hbase4.apache.org,38271,1679889217353 2023-03-27 03:53:39,419 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38271%2C1679889217353, suffix=, logDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/MasterData/WALs/jenkins-hbase4.apache.org,38271,1679889217353, archiveDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/MasterData/oldWALs, maxLogs=10 2023-03-27 03:53:39,471 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34783,DS-0a811c54-aac2-43e0-90a5-2921a697cf04,DISK] 2023-03-27 03:53:39,480 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-03-27 03:53:39,552 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/MasterData/WALs/jenkins-hbase4.apache.org,38271,1679889217353/jenkins-hbase4.apache.org%2C38271%2C1679889217353.1679889219429 2023-03-27 03:53:39,552 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34783,DS-0a811c54-aac2-43e0-90a5-2921a697cf04,DISK]] 2023-03-27 03:53:39,553 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-03-27 03:53:39,554 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:53:39,558 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-03-27 03:53:39,559 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-03-27 03:53:39,631 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-03-27 03:53:39,638 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-03-27 03:53:39,662 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-03-27 03:53:39,676 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:53:39,683 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-27 03:53:39,685 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-03-27 03:53:39,702 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-03-27 03:53:39,706 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:53:39,708 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72724958, jitterRate=0.08368632197380066}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-27 03:53:39,708 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-27 03:53:39,709 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-03-27 03:53:39,731 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-03-27 03:53:39,731 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-03-27 03:53:39,734 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-03-27 03:53:39,736 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-03-27 03:53:39,769 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 32 msec 2023-03-27 03:53:39,769 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-03-27 03:53:39,801 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-03-27 03:53:39,808 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-03-27 03:53:39,834 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-03-27 03:53:39,838 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-03-27 03:53:39,840 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-03-27 03:53:39,845 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-03-27 03:53:39,849 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-03-27 03:53:39,853 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-27 03:53:39,855 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-03-27 03:53:39,855 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-03-27 03:53:39,867 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-03-27 03:53:39,871 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-27 03:53:39,871 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-03-27 03:53:39,871 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-27 03:53:39,872 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,38271,1679889217353, sessionid=0x101241b1daf0000, setting cluster-up flag (Was=false) 2023-03-27 03:53:39,886 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-27 03:53:39,893 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-03-27 03:53:39,894 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38271,1679889217353 2023-03-27 03:53:39,900 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-27 03:53:39,905 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-03-27 03:53:39,906 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38271,1679889217353 2023-03-27 03:53:39,909 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.hbase-snapshot/.tmp 2023-03-27 03:53:40,009 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(952): ClusterId : 54847249-82a7-4951-a85d-e1357ad5c94f 2023-03-27 03:53:40,014 DEBUG [RS:0;jenkins-hbase4:39727] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-03-27 03:53:40,022 DEBUG [RS:0;jenkins-hbase4:39727] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-03-27 03:53:40,022 DEBUG [RS:0;jenkins-hbase4:39727] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-03-27 03:53:40,025 DEBUG [RS:0;jenkins-hbase4:39727] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-03-27 03:53:40,026 DEBUG [RS:0;jenkins-hbase4:39727] zookeeper.ReadOnlyZKClient(139): Connect 0x5c4e655e to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:53:40,031 DEBUG [RS:0;jenkins-hbase4:39727] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@528980a7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:53:40,032 DEBUG [RS:0;jenkins-hbase4:39727] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@56b60e2e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-03-27 03:53:40,039 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-03-27 03:53:40,053 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-27 03:53:40,053 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-27 03:53:40,053 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-27 03:53:40,054 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-03-27 03:53:40,054 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-03-27 03:53:40,054 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-27 03:53:40,054 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-03-27 03:53:40,054 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-27 03:53:40,061 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1679889250061 2023-03-27 03:53:40,064 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-03-27 03:53:40,066 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-03-27 03:53:40,066 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-03-27 03:53:40,067 DEBUG [RS:0;jenkins-hbase4:39727] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:39727 2023-03-27 03:53:40,071 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-27 03:53:40,072 INFO [RS:0;jenkins-hbase4:39727] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-03-27 03:53:40,072 INFO [RS:0;jenkins-hbase4:39727] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-03-27 03:53:40,072 DEBUG [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(1023): About to register with Master. 2023-03-27 03:53:40,076 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,38271,1679889217353 with isa=jenkins-hbase4.apache.org/172.31.14.131:39727, startcode=1679889218560 2023-03-27 03:53:40,080 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-03-27 03:53:40,089 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-03-27 03:53:40,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-03-27 03:53:40,090 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-03-27 03:53:40,091 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-03-27 03:53:40,092 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-27 03:53:40,097 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-03-27 03:53:40,100 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-03-27 03:53:40,101 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-03-27 03:53:40,101 DEBUG [RS:0;jenkins-hbase4:39727] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-03-27 03:53:40,116 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-03-27 03:53:40,117 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-03-27 03:53:40,120 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-27 03:53:40,121 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-03-27 03:53:40,121 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001 2023-03-27 03:53:40,125 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1679889220119,5,FailOnTimeoutGroup] 2023-03-27 03:53:40,125 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1679889220125,5,FailOnTimeoutGroup] 2023-03-27 03:53:40,125 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-03-27 03:53:40,125 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-03-27 03:53:40,127 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-03-27 03:53:40,127 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-03-27 03:53:40,181 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51665, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-03-27 03:53:40,191 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:40,208 DEBUG [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001 2023-03-27 03:53:40,208 DEBUG [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:43651 2023-03-27 03:53:40,208 DEBUG [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-03-27 03:53:40,215 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-27 03:53:40,215 DEBUG [RS:0;jenkins-hbase4:39727] zookeeper.ZKUtil(162): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:40,216 WARN [RS:0;jenkins-hbase4:39727] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-03-27 03:53:40,220 INFO [RS:0;jenkins-hbase4:39727] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-27 03:53:40,220 DEBUG [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(1947): logDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/WALs/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:40,221 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,39727,1679889218560] 2023-03-27 03:53:40,234 DEBUG [RS:0;jenkins-hbase4:39727] zookeeper.ZKUtil(162): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:40,248 DEBUG [RS:0;jenkins-hbase4:39727] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-03-27 03:53:40,258 INFO [RS:0;jenkins-hbase4:39727] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-03-27 03:53:40,278 INFO [RS:0;jenkins-hbase4:39727] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-03-27 03:53:40,282 INFO [RS:0;jenkins-hbase4:39727] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-27 03:53:40,282 INFO [RS:0;jenkins-hbase4:39727] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-27 03:53:40,283 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-03-27 03:53:40,292 INFO [RS:0;jenkins-hbase4:39727] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-03-27 03:53:40,292 DEBUG [RS:0;jenkins-hbase4:39727] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-27 03:53:40,293 DEBUG [RS:0;jenkins-hbase4:39727] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-27 03:53:40,293 DEBUG [RS:0;jenkins-hbase4:39727] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-27 03:53:40,293 DEBUG [RS:0;jenkins-hbase4:39727] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-27 03:53:40,293 DEBUG [RS:0;jenkins-hbase4:39727] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-27 03:53:40,294 DEBUG [RS:0;jenkins-hbase4:39727] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-03-27 03:53:40,294 DEBUG [RS:0;jenkins-hbase4:39727] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-27 03:53:40,294 DEBUG [RS:0;jenkins-hbase4:39727] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-27 03:53:40,294 DEBUG [RS:0;jenkins-hbase4:39727] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-27 03:53:40,294 DEBUG [RS:0;jenkins-hbase4:39727] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-03-27 03:53:40,296 INFO [RS:0;jenkins-hbase4:39727] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-03-27 03:53:40,296 INFO [RS:0;jenkins-hbase4:39727] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-03-27 03:53:40,296 INFO [RS:0;jenkins-hbase4:39727] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-03-27 03:53:40,313 INFO [RS:0;jenkins-hbase4:39727] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-03-27 03:53:40,315 INFO [RS:0;jenkins-hbase4:39727] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39727,1679889218560-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-27 03:53:40,332 INFO [RS:0;jenkins-hbase4:39727] regionserver.Replication(203): jenkins-hbase4.apache.org,39727,1679889218560 started 2023-03-27 03:53:40,333 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,39727,1679889218560, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:39727, sessionid=0x101241b1daf0001 2023-03-27 03:53:40,333 DEBUG [RS:0;jenkins-hbase4:39727] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-03-27 03:53:40,333 DEBUG [RS:0;jenkins-hbase4:39727] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:40,333 DEBUG [RS:0;jenkins-hbase4:39727] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39727,1679889218560' 2023-03-27 03:53:40,333 DEBUG [RS:0;jenkins-hbase4:39727] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:40,334 DEBUG [RS:0;jenkins-hbase4:39727] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:40,335 DEBUG [RS:0;jenkins-hbase4:39727] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-03-27 03:53:40,335 DEBUG [RS:0;jenkins-hbase4:39727] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-03-27 03:53:40,335 DEBUG [RS:0;jenkins-hbase4:39727] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:40,335 DEBUG [RS:0;jenkins-hbase4:39727] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39727,1679889218560' 2023-03-27 03:53:40,335 DEBUG [RS:0;jenkins-hbase4:39727] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-27 03:53:40,336 DEBUG [RS:0;jenkins-hbase4:39727] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-27 03:53:40,336 DEBUG [RS:0;jenkins-hbase4:39727] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-03-27 03:53:40,336 INFO [RS:0;jenkins-hbase4:39727] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-03-27 03:53:40,336 INFO [RS:0;jenkins-hbase4:39727] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-03-27 03:53:40,447 INFO [RS:0;jenkins-hbase4:39727] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39727%2C1679889218560, suffix=, logDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/WALs/jenkins-hbase4.apache.org,39727,1679889218560, archiveDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/oldWALs, maxLogs=32 2023-03-27 03:53:40,465 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34783,DS-0a811c54-aac2-43e0-90a5-2921a697cf04,DISK] 2023-03-27 03:53:40,469 INFO [RS:0;jenkins-hbase4:39727] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/WALs/jenkins-hbase4.apache.org,39727,1679889218560/jenkins-hbase4.apache.org%2C39727%2C1679889218560.1679889220449 2023-03-27 03:53:40,469 DEBUG [RS:0;jenkins-hbase4:39727] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34783,DS-0a811c54-aac2-43e0-90a5-2921a697cf04,DISK]] 2023-03-27 03:53:40,555 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:53:40,557 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-27 03:53:40,560 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/info 2023-03-27 03:53:40,561 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-27 03:53:40,562 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:53:40,562 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-27 03:53:40,565 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/rep_barrier 2023-03-27 03:53:40,565 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-27 03:53:40,566 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:53:40,566 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-27 03:53:40,568 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/table 2023-03-27 03:53:40,569 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-27 03:53:40,570 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:53:40,571 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740 2023-03-27 03:53:40,572 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740 2023-03-27 03:53:40,575 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-27 03:53:40,577 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-27 03:53:40,580 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:53:40,581 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70782160, jitterRate=0.05473637580871582}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-27 03:53:40,581 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-27 03:53:40,581 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-27 03:53:40,582 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-27 03:53:40,582 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-27 03:53:40,582 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-27 03:53:40,582 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-27 03:53:40,583 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-27 03:53:40,583 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-27 03:53:40,588 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-03-27 03:53:40,588 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-03-27 03:53:40,598 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-03-27 03:53:40,610 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-03-27 03:53:40,612 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-03-27 03:53:40,764 DEBUG [jenkins-hbase4:38271] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-03-27 03:53:40,767 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,39727,1679889218560, state=OPENING 2023-03-27 03:53:40,812 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-03-27 03:53:40,904 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-27 03:53:40,905 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-27 03:53:40,907 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:53:41,092 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:41,095 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-27 03:53:41,099 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52968, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-27 03:53:41,111 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-03-27 03:53:41,112 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-03-27 03:53:41,115 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39727%2C1679889218560.meta, suffix=.meta, logDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/WALs/jenkins-hbase4.apache.org,39727,1679889218560, archiveDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/oldWALs, maxLogs=32 2023-03-27 03:53:41,134 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34783,DS-0a811c54-aac2-43e0-90a5-2921a697cf04,DISK] 2023-03-27 03:53:41,138 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/WALs/jenkins-hbase4.apache.org,39727,1679889218560/jenkins-hbase4.apache.org%2C39727%2C1679889218560.meta.1679889221116.meta 2023-03-27 03:53:41,139 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34783,DS-0a811c54-aac2-43e0-90a5-2921a697cf04,DISK]] 2023-03-27 03:53:41,139 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-03-27 03:53:41,141 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-03-27 03:53:41,157 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-03-27 03:53:41,161 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-03-27 03:53:41,167 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-03-27 03:53:41,167 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:53:41,167 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-03-27 03:53:41,167 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-03-27 03:53:41,170 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-03-27 03:53:41,172 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/info 2023-03-27 03:53:41,172 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/info 2023-03-27 03:53:41,172 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-03-27 03:53:41,173 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:53:41,173 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-03-27 03:53:41,175 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/rep_barrier 2023-03-27 03:53:41,175 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/rep_barrier 2023-03-27 03:53:41,175 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-03-27 03:53:41,176 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:53:41,176 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-03-27 03:53:41,178 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/table 2023-03-27 03:53:41,178 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/table 2023-03-27 03:53:41,178 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-03-27 03:53:41,179 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:53:41,181 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740 2023-03-27 03:53:41,183 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740 2023-03-27 03:53:41,186 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-03-27 03:53:41,188 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-03-27 03:53:41,190 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74040621, jitterRate=0.10329122841358185}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-03-27 03:53:41,190 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-03-27 03:53:41,199 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1679889221083 2023-03-27 03:53:41,216 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-03-27 03:53:41,217 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-03-27 03:53:41,217 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,39727,1679889218560, state=OPEN 2023-03-27 03:53:41,220 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-03-27 03:53:41,220 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-03-27 03:53:41,224 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-03-27 03:53:41,224 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,39727,1679889218560 in 313 msec 2023-03-27 03:53:41,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-03-27 03:53:41,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 628 msec 2023-03-27 03:53:41,233 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.2800 sec 2023-03-27 03:53:41,233 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1679889221233, completionTime=-1 2023-03-27 03:53:41,234 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-03-27 03:53:41,234 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-03-27 03:53:41,295 DEBUG [hconnection-0x3771e5eb-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-27 03:53:41,299 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52978, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-27 03:53:41,317 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-03-27 03:53:41,317 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1679889281317 2023-03-27 03:53:41,317 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1679889341317 2023-03-27 03:53:41,317 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 83 msec 2023-03-27 03:53:41,339 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38271,1679889217353-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-03-27 03:53:41,340 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38271,1679889217353-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-27 03:53:41,340 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38271,1679889217353-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-03-27 03:53:41,342 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:38271, period=300000, unit=MILLISECONDS is enabled. 2023-03-27 03:53:41,342 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-03-27 03:53:41,348 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-03-27 03:53:41,356 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-03-27 03:53:41,358 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-03-27 03:53:41,367 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-03-27 03:53:41,369 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-03-27 03:53:41,372 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-27 03:53:41,394 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/hbase/namespace/08f8d4ce92d3daa28bf960c05f8fc02f 2023-03-27 03:53:41,396 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/hbase/namespace/08f8d4ce92d3daa28bf960c05f8fc02f empty. 2023-03-27 03:53:41,397 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/hbase/namespace/08f8d4ce92d3daa28bf960c05f8fc02f 2023-03-27 03:53:41,397 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-03-27 03:53:41,419 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-03-27 03:53:41,421 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 08f8d4ce92d3daa28bf960c05f8fc02f, NAME => 'hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp 2023-03-27 03:53:41,433 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:53:41,433 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 08f8d4ce92d3daa28bf960c05f8fc02f, disabling compactions & flushes 2023-03-27 03:53:41,433 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f. 2023-03-27 03:53:41,433 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f. 2023-03-27 03:53:41,433 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f. after waiting 0 ms 2023-03-27 03:53:41,433 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f. 2023-03-27 03:53:41,433 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f. 2023-03-27 03:53:41,433 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 08f8d4ce92d3daa28bf960c05f8fc02f: 2023-03-27 03:53:41,438 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-03-27 03:53:41,452 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1679889221440"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889221440"}]},"ts":"1679889221440"} 2023-03-27 03:53:41,486 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-27 03:53:41,488 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-27 03:53:41,493 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889221488"}]},"ts":"1679889221488"} 2023-03-27 03:53:41,498 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-03-27 03:53:41,512 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=08f8d4ce92d3daa28bf960c05f8fc02f, ASSIGN}] 2023-03-27 03:53:41,515 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=08f8d4ce92d3daa28bf960c05f8fc02f, ASSIGN 2023-03-27 03:53:41,517 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=08f8d4ce92d3daa28bf960c05f8fc02f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=false 2023-03-27 03:53:41,668 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=08f8d4ce92d3daa28bf960c05f8fc02f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:41,668 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1679889221668"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889221668"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889221668"}]},"ts":"1679889221668"} 2023-03-27 03:53:41,674 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 08f8d4ce92d3daa28bf960c05f8fc02f, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:53:41,834 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f. 2023-03-27 03:53:41,835 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 08f8d4ce92d3daa28bf960c05f8fc02f, NAME => 'hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f.', STARTKEY => '', ENDKEY => ''} 2023-03-27 03:53:41,836 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 08f8d4ce92d3daa28bf960c05f8fc02f 2023-03-27 03:53:41,836 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:53:41,836 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 08f8d4ce92d3daa28bf960c05f8fc02f 2023-03-27 03:53:41,836 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 08f8d4ce92d3daa28bf960c05f8fc02f 2023-03-27 03:53:41,838 INFO [StoreOpener-08f8d4ce92d3daa28bf960c05f8fc02f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 08f8d4ce92d3daa28bf960c05f8fc02f 2023-03-27 03:53:41,841 DEBUG [StoreOpener-08f8d4ce92d3daa28bf960c05f8fc02f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/namespace/08f8d4ce92d3daa28bf960c05f8fc02f/info 2023-03-27 03:53:41,841 DEBUG [StoreOpener-08f8d4ce92d3daa28bf960c05f8fc02f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/namespace/08f8d4ce92d3daa28bf960c05f8fc02f/info 2023-03-27 03:53:41,841 INFO [StoreOpener-08f8d4ce92d3daa28bf960c05f8fc02f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 08f8d4ce92d3daa28bf960c05f8fc02f columnFamilyName info 2023-03-27 03:53:41,842 INFO [StoreOpener-08f8d4ce92d3daa28bf960c05f8fc02f-1] regionserver.HStore(310): Store=08f8d4ce92d3daa28bf960c05f8fc02f/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:53:41,843 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/namespace/08f8d4ce92d3daa28bf960c05f8fc02f 2023-03-27 03:53:41,844 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/namespace/08f8d4ce92d3daa28bf960c05f8fc02f 2023-03-27 03:53:41,849 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 08f8d4ce92d3daa28bf960c05f8fc02f 2023-03-27 03:53:41,853 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/namespace/08f8d4ce92d3daa28bf960c05f8fc02f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:53:41,854 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 08f8d4ce92d3daa28bf960c05f8fc02f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59372055, jitterRate=-0.11528743803501129}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-27 03:53:41,854 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 08f8d4ce92d3daa28bf960c05f8fc02f: 2023-03-27 03:53:41,856 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f., pid=6, masterSystemTime=1679889221828 2023-03-27 03:53:41,859 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f. 2023-03-27 03:53:41,859 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f. 2023-03-27 03:53:41,861 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=08f8d4ce92d3daa28bf960c05f8fc02f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:41,861 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1679889221860"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889221860"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889221860"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889221860"}]},"ts":"1679889221860"} 2023-03-27 03:53:41,868 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-03-27 03:53:41,868 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 08f8d4ce92d3daa28bf960c05f8fc02f, server=jenkins-hbase4.apache.org,39727,1679889218560 in 190 msec 2023-03-27 03:53:41,871 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-03-27 03:53:41,871 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=08f8d4ce92d3daa28bf960c05f8fc02f, ASSIGN in 356 msec 2023-03-27 03:53:41,872 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-27 03:53:41,873 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889221872"}]},"ts":"1679889221872"} 2023-03-27 03:53:41,875 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-03-27 03:53:41,878 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-03-27 03:53:41,881 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 520 msec 2023-03-27 03:53:41,970 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-03-27 03:53:41,972 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-03-27 03:53:41,972 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-27 03:53:42,009 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-03-27 03:53:42,026 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-27 03:53:42,032 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 29 msec 2023-03-27 03:53:42,043 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-03-27 03:53:42,054 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-27 03:53:42,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 17 msec 2023-03-27 03:53:42,069 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-03-27 03:53:42,072 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-03-27 03:53:42,072 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 3.432sec 2023-03-27 03:53:42,074 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-03-27 03:53:42,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-03-27 03:53:42,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-03-27 03:53:42,077 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38271,1679889217353-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-03-27 03:53:42,077 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38271,1679889217353-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-03-27 03:53:42,087 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-03-27 03:53:42,114 DEBUG [Listener at localhost/36131] zookeeper.ReadOnlyZKClient(139): Connect 0x18f0808a to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:53:42,118 DEBUG [Listener at localhost/36131] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5759ba4d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:53:42,131 DEBUG [hconnection-0x1911755b-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-27 03:53:42,142 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52994, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-27 03:53:42,152 INFO [Listener at localhost/36131] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,38271,1679889217353 2023-03-27 03:53:42,154 DEBUG [Listener at localhost/36131] zookeeper.ReadOnlyZKClient(139): Connect 0x69d78ff9 to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:53:42,160 DEBUG [Listener at localhost/36131] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@47652242, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:53:42,161 INFO [Listener at localhost/36131] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-27 03:53:42,183 INFO [Listener at localhost/36131] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=212, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=381, ProcessCount=170, AvailableMemoryMB=9519 2023-03-27 03:53:42,193 DEBUG [Listener at localhost/36131] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-27 03:53:42,224 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40004, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-27 03:53:42,239 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-27 03:53:42,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:42,243 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-27 03:53:42,245 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-27 03:53:42,247 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-03-27 03:53:42,248 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404 2023-03-27 03:53:42,249 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404 empty. 2023-03-27 03:53:42,251 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404 2023-03-27 03:53:42,251 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-03-27 03:53:42,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-27 03:53:42,361 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-27 03:53:42,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-27 03:53:42,668 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-27 03:53:42,670 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 647148480c2004d294d48558b731b404, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp 2023-03-27 03:53:42,683 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:53:42,683 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1604): Closing 647148480c2004d294d48558b731b404, disabling compactions & flushes 2023-03-27 03:53:42,683 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:42,684 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:42,684 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. after waiting 0 ms 2023-03-27 03:53:42,684 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:42,684 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:42,684 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1558): Region close journal for 647148480c2004d294d48558b731b404: 2023-03-27 03:53:42,688 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-03-27 03:53:42,690 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679889222689"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889222689"}]},"ts":"1679889222689"} 2023-03-27 03:53:42,692 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-27 03:53:42,694 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-27 03:53:42,694 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889222694"}]},"ts":"1679889222694"} 2023-03-27 03:53:42,697 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-03-27 03:53:42,701 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=647148480c2004d294d48558b731b404, ASSIGN}] 2023-03-27 03:53:42,703 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=647148480c2004d294d48558b731b404, ASSIGN 2023-03-27 03:53:42,704 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=647148480c2004d294d48558b731b404, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=false 2023-03-27 03:53:42,856 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=647148480c2004d294d48558b731b404, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:42,856 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679889222855"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889222855"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889222855"}]},"ts":"1679889222855"} 2023-03-27 03:53:42,859 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 647148480c2004d294d48558b731b404, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:53:42,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-27 03:53:43,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:43,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 647148480c2004d294d48558b731b404, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404.', STARTKEY => '', ENDKEY => ''} 2023-03-27 03:53:43,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 647148480c2004d294d48558b731b404 2023-03-27 03:53:43,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:53:43,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 647148480c2004d294d48558b731b404 2023-03-27 03:53:43,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 647148480c2004d294d48558b731b404 2023-03-27 03:53:43,024 INFO [StoreOpener-647148480c2004d294d48558b731b404-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 647148480c2004d294d48558b731b404 2023-03-27 03:53:43,026 DEBUG [StoreOpener-647148480c2004d294d48558b731b404-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1 2023-03-27 03:53:43,027 DEBUG [StoreOpener-647148480c2004d294d48558b731b404-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1 2023-03-27 03:53:43,030 INFO [StoreOpener-647148480c2004d294d48558b731b404-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 647148480c2004d294d48558b731b404 columnFamilyName family1 2023-03-27 03:53:43,033 INFO [StoreOpener-647148480c2004d294d48558b731b404-1] regionserver.HStore(310): Store=647148480c2004d294d48558b731b404/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:53:43,036 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404 2023-03-27 03:53:43,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404 2023-03-27 03:53:43,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 647148480c2004d294d48558b731b404 2023-03-27 03:53:43,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:53:43,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 647148480c2004d294d48558b731b404; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61564516, jitterRate=-0.08261722326278687}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-27 03:53:43,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 647148480c2004d294d48558b731b404: 2023-03-27 03:53:43,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404., pid=11, masterSystemTime=1679889223013 2023-03-27 03:53:43,055 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:43,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:43,056 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=647148480c2004d294d48558b731b404, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:43,056 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679889223056"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889223056"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889223056"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889223056"}]},"ts":"1679889223056"} 2023-03-27 03:53:43,063 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-03-27 03:53:43,063 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 647148480c2004d294d48558b731b404, server=jenkins-hbase4.apache.org,39727,1679889218560 in 201 msec 2023-03-27 03:53:43,067 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-03-27 03:53:43,068 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=647148480c2004d294d48558b731b404, ASSIGN in 362 msec 2023-03-27 03:53:43,069 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-27 03:53:43,069 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889223069"}]},"ts":"1679889223069"} 2023-03-27 03:53:43,072 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-03-27 03:53:43,078 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-03-27 03:53:43,081 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 839 msec 2023-03-27 03:53:43,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-03-27 03:53:43,369 INFO [Listener at localhost/36131] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-03-27 03:53:43,387 DEBUG [hconnection-0x23d918c4-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-27 03:53:43,391 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60820, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-27 03:53:43,545 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:53:43,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:43,552 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-27 03:53:43,553 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:43,553 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-27 03:53:43,553 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-27 03:53:43,554 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:43,554 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:43,555 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:43,555 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:43,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:43,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:43,556 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:43,556 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:53:43,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:43,556 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:43,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:53:43,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:43,557 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:43,559 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:43,559 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-27 03:53:43,559 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:43,560 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-27 03:53:43,561 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:53:43,561 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:53:43,561 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:43,562 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. started... 2023-03-27 03:53:43,563 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 647148480c2004d294d48558b731b404 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-27 03:53:44,101 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130395dd07d71db40279562deedffba0853 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130395dd07d71db40279562deedffba0853 2023-03-27 03:53:44,519 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/2ef373ea86eb4b0581d6e14e68800bf9 2023-03-27 03:53:44,531 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/2ef373ea86eb4b0581d6e14e68800bf9 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/2ef373ea86eb4b0581d6e14e68800bf9 2023-03-27 03:53:44,541 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/2ef373ea86eb4b0581d6e14e68800bf9, entries=1000, sequenceid=5, filesize=175.8 K 2023-03-27 03:53:44,544 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 647148480c2004d294d48558b731b404 in 982ms, sequenceid=5, compaction requested=false 2023-03-27 03:53:44,546 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-27 03:53:44,550 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 647148480c2004d294d48558b731b404: 2023-03-27 03:53:44,550 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:44,550 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:53:44,551 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:53:44,551 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:44,551 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-27 03:53:44,551 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-27 03:53:44,553 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:44,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:44,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:53:44,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:44,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:44,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:44,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:44,557 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-27 03:53:44,557 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-27 03:53:44,557 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5bb974df[Count = 0] remaining members to acquire global barrier 2023-03-27 03:53:44,557 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,558 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,559 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-27 03:53:44,559 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-27 03:53:44,559 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:53:44,559 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:44,559 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:53:44,563 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-27 03:53:44,563 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:44,563 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:44,563 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-27 03:53:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:44,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:44,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:44,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:44,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:44,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:53:44,568 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-27 03:53:44,568 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:53:44,568 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-27 03:53:44,568 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:53:44,568 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,568 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:53:44,571 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,571 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,571 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:44,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:44,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:44,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:44,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:44,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:44,572 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:44,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:44,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:44,574 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:44,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:44,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:44,582 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:44,582 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:44,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:44,583 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:44,582 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:44,583 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-27 03:53:44,583 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:53:44,583 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,583 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:44,583 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:44,584 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,584 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,584 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:44,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:44,585 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-27 03:53:44,586 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:53:44,686 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:53:44,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:53:44,712 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:53:44,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-27 03:53:44,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,716 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-27 03:53:44,716 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:44,716 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-27 03:53:44,716 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-27 03:53:44,718 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,718 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,720 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:44,721 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:44,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:44,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:44,722 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:44,722 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:53:44,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,723 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:53:44,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,723 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,734 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-27 03:53:44,734 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:44,734 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-27 03:53:44,735 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:44,735 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-27 03:53:44,740 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:53:44,740 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:53:44,740 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:44,741 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. started... 2023-03-27 03:53:44,741 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 647148480c2004d294d48558b731b404 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-27 03:53:45,198 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128512816e53b204d04859e60220f63aa23 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128512816e53b204d04859e60220f63aa23 2023-03-27 03:53:45,658 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/6f21901c2e534fc0b23391346b83006e 2023-03-27 03:53:45,671 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/6f21901c2e534fc0b23391346b83006e as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/6f21901c2e534fc0b23391346b83006e 2023-03-27 03:53:45,682 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/6f21901c2e534fc0b23391346b83006e, entries=1, sequenceid=9, filesize=5.0 K 2023-03-27 03:53:45,684 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 647148480c2004d294d48558b731b404 in 943ms, sequenceid=9, compaction requested=false 2023-03-27 03:53:45,684 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 647148480c2004d294d48558b731b404: 2023-03-27 03:53:45,684 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:45,684 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:53:45,685 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:53:45,685 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,685 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-27 03:53:45,685 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-27 03:53:45,689 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,689 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:45,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:45,689 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,690 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:53:45,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:45,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:45,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:45,695 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-27 03:53:45,695 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35da58e6[Count = 0] remaining members to acquire global barrier 2023-03-27 03:53:45,695 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-27 03:53:45,695 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,696 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,697 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-27 03:53:45,697 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-27 03:53:45,697 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:53:45,697 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,697 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:53:45,701 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-27 03:53:45,701 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,701 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:45,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,701 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-27 03:53:45,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:45,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:45,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:45,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:45,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:45,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:53:45,707 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-27 03:53:45,707 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-27 03:53:45,708 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:53:45,708 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:53:45,708 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,708 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:53:45,713 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,713 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,713 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:45,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:45,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:45,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:45,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:45,715 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,717 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:45,722 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,722 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,722 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,722 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:45,722 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:45,722 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:45,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:45,722 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,722 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,723 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,723 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:45,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:45,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-27 03:53:45,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:53:45,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:45,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:45,723 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,724 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-27 03:53:45,724 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:53:45,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:45,726 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-27 03:53:45,824 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:53:45,827 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:53:45,838 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:53:45,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-27 03:53:45,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,846 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-27 03:53:45,846 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:45,847 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-27 03:53:45,847 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-27 03:53:45,848 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,848 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,850 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:45,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:45,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:45,850 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,851 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,851 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:53:45,851 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:53:45,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,852 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,852 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-27 03:53:45,852 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,852 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-27 03:53:45,852 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:45,854 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-27 03:53:45,854 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:53:45,854 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:53:45,856 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:45,856 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. started... 2023-03-27 03:53:45,856 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 647148480c2004d294d48558b731b404 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:53:45,897 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512013864ea74254c40469996786d8f8e661b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512013864ea74254c40469996786d8f8e661b 2023-03-27 03:53:45,931 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/10dbc5a8cd9e471a83a58d7661ce493e 2023-03-27 03:53:45,942 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/10dbc5a8cd9e471a83a58d7661ce493e as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/10dbc5a8cd9e471a83a58d7661ce493e 2023-03-27 03:53:45,953 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/10dbc5a8cd9e471a83a58d7661ce493e, entries=1, sequenceid=13, filesize=5.0 K 2023-03-27 03:53:45,954 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 647148480c2004d294d48558b731b404 in 98ms, sequenceid=13, compaction requested=true 2023-03-27 03:53:45,954 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 647148480c2004d294d48558b731b404: 2023-03-27 03:53:45,954 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:45,954 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:53:45,955 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:53:45,955 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,955 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-27 03:53:45,955 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-27 03:53:45,958 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,958 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:45,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:45,958 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,958 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:53:45,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:45,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:45,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:45,961 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-27 03:53:45,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3ada6437[Count = 0] remaining members to acquire global barrier 2023-03-27 03:53:45,962 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-27 03:53:45,962 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,965 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,965 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-27 03:53:45,965 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-27 03:53:45,965 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:53:45,965 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,965 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:53:45,967 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,967 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-27 03:53:45,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:45,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:45,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:45,967 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:45,969 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-27 03:53:45,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:45,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:45,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:53:45,974 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-27 03:53:45,974 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-27 03:53:45,974 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:53:45,974 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:53:45,974 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,974 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:53:45,977 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,977 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:45,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,977 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:45,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:45,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:45,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:45,978 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:45,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:45,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,980 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:45,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,988 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,988 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,988 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:45,988 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:45,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:45,988 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:45,989 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:45,989 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,989 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,989 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:45,989 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-27 03:53:45,989 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-27 03:53:45,989 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:53:45,989 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:45,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:45,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:45,991 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-27 03:53:45,991 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:53:46,091 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:53:46,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:53:46,112 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:53:46,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-27 03:53:46,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,115 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-27 03:53:46,116 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:46,116 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-27 03:53:46,116 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-27 03:53:46,117 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,117 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,122 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:46,122 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:46,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:46,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:46,122 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:46,122 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:53:46,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,123 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:53:46,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,124 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,136 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-27 03:53:46,136 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,136 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-27 03:53:46,136 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:46,141 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-27 03:53:46,144 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:53:46,144 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:53:46,144 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:46,144 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. started... 2023-03-27 03:53:46,144 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 647148480c2004d294d48558b731b404 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:53:46,605 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512052a5ccb7fc1044ef4821b3c0e81b5c31e to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512052a5ccb7fc1044ef4821b3c0e81b5c31e 2023-03-27 03:53:46,629 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/6da4899d3a4b44e49a3a88ea2f11ff41 2023-03-27 03:53:46,646 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/6da4899d3a4b44e49a3a88ea2f11ff41 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/6da4899d3a4b44e49a3a88ea2f11ff41 2023-03-27 03:53:46,658 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/6da4899d3a4b44e49a3a88ea2f11ff41, entries=1, sequenceid=17, filesize=5.0 K 2023-03-27 03:53:46,659 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 647148480c2004d294d48558b731b404 in 515ms, sequenceid=17, compaction requested=true 2023-03-27 03:53:46,660 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 647148480c2004d294d48558b731b404: 2023-03-27 03:53:46,661 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:46,661 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:53:46,661 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:53:46,662 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:46,662 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-27 03:53:46,662 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-27 03:53:46,664 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,664 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:46,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:46,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:46,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:46,665 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,665 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:53:46,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:46,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:46,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:46,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:46,671 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-27 03:53:46,671 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@778f1f08[Count = 0] remaining members to acquire global barrier 2023-03-27 03:53:46,671 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-27 03:53:46,671 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,672 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,673 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-27 03:53:46,673 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-27 03:53:46,673 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:53:46,673 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:46,673 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:53:46,675 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-27 03:53:46,675 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:46,676 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-27 03:53:46,676 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:46,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:46,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:46,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:46,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:46,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:46,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:46,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:46,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:46,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:53:46,687 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-27 03:53:46,687 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-27 03:53:46,687 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:53:46,687 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:53:46,687 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,687 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:53:46,691 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,691 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:46,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:46,691 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:46,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:46,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:46,692 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:46,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:46,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:46,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,695 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:46,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:46,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:46,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:46,707 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:46,707 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:46,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:46,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:46,708 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:46,708 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-27 03:53:46,708 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:53:46,708 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-27 03:53:46,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:46,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:46,713 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:46,714 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,714 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,714 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:46,714 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,714 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,714 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-27 03:53:46,714 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:53:46,714 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,738 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-27 03:53:46,814 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:53:46,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:53:46,827 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:53:46,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-27 03:53:46,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,852 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-27 03:53:46,852 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:46,853 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-27 03:53:46,853 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-27 03:53:46,854 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,854 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,855 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:46,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:46,856 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:46,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:46,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,859 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-03-27 03:53:46,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:53:46,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,859 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,859 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:46,859 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:53:46,859 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-27 03:53:46,859 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:46,860 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-27 03:53:46,860 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:46,860 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-27 03:53:46,860 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-03-27 03:53:46,861 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:53:46,861 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:53:46,861 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:46,861 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. started... 2023-03-27 03:53:46,861 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 647148480c2004d294d48558b731b404 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:53:46,927 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512283ce26f2981764e8caa21c7d2a830f5cb to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512283ce26f2981764e8caa21c7d2a830f5cb 2023-03-27 03:53:46,969 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/eed40470ce27436ba67812f33572b49c 2023-03-27 03:53:46,982 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/eed40470ce27436ba67812f33572b49c as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/eed40470ce27436ba67812f33572b49c 2023-03-27 03:53:46,994 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/eed40470ce27436ba67812f33572b49c, entries=1, sequenceid=21, filesize=5.0 K 2023-03-27 03:53:46,996 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 647148480c2004d294d48558b731b404 in 135ms, sequenceid=21, compaction requested=true 2023-03-27 03:53:46,996 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 647148480c2004d294d48558b731b404: 2023-03-27 03:53:46,996 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:46,996 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:53:46,998 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:53:46,998 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:46,998 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-27 03:53:46,998 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-27 03:53:47,004 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,004 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:47,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:47,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:47,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:47,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,005 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,007 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:53:47,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:47,008 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-27 03:53:47,008 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@678082aa[Count = 0] remaining members to acquire global barrier 2023-03-27 03:53:47,008 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-27 03:53:47,008 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,010 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,011 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-27 03:53:47,011 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-27 03:53:47,011 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:53:47,011 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,011 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:53:47,013 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,013 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-27 03:53:47,013 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:47,013 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-27 03:53:47,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:47,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:47,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:47,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:47,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:47,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:53:47,019 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-27 03:53:47,019 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-27 03:53:47,019 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:53:47,019 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:53:47,019 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,019 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:53:47,021 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,021 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:47,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,022 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:47,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:47,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:47,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:47,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:47,024 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:47,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,026 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:47,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,032 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,032 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:47,032 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-27 03:53:47,033 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,033 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,033 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,033 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,033 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,033 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-27 03:53:47,033 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:53:47,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:47,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:47,034 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-27 03:53:47,034 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:53:47,033 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:47,033 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:47,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:47,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:47,134 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:53:47,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:53:47,147 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:53:47,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-27 03:53:47,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,164 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-27 03:53:47,164 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:47,170 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-27 03:53:47,170 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-27 03:53:47,170 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,170 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,172 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:47,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:47,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:47,173 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,173 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,173 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:53:47,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,174 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:53:47,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,175 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,175 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-27 03:53:47,175 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,175 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-27 03:53:47,175 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:47,176 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-27 03:53:47,176 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:53:47,176 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:53:47,176 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:47,177 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. started... 2023-03-27 03:53:47,177 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 647148480c2004d294d48558b731b404 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:53:47,275 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512316e9e88ee92a14b30b09381e7bf49bd93 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512316e9e88ee92a14b30b09381e7bf49bd93 2023-03-27 03:53:47,305 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/95f27c856fc14ce7b7c4a32a0ebe82fc 2023-03-27 03:53:47,318 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/95f27c856fc14ce7b7c4a32a0ebe82fc as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/95f27c856fc14ce7b7c4a32a0ebe82fc 2023-03-27 03:53:47,329 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/95f27c856fc14ce7b7c4a32a0ebe82fc, entries=1, sequenceid=25, filesize=5.0 K 2023-03-27 03:53:47,331 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 647148480c2004d294d48558b731b404 in 154ms, sequenceid=25, compaction requested=true 2023-03-27 03:53:47,331 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 647148480c2004d294d48558b731b404: 2023-03-27 03:53:47,331 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:47,331 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:53:47,332 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:53:47,332 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,332 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-27 03:53:47,332 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-27 03:53:47,334 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,334 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:47,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:47,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:47,335 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,335 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:53:47,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:47,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:47,338 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-27 03:53:47,338 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7afbd4[Count = 0] remaining members to acquire global barrier 2023-03-27 03:53:47,338 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-27 03:53:47,338 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,339 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,340 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-27 03:53:47,340 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-27 03:53:47,340 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:53:47,340 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,340 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:53:47,343 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-27 03:53:47,343 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:47,343 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-27 03:53:47,343 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:47,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:47,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:47,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:47,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:47,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:53:47,349 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-27 03:53:47,349 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-27 03:53:47,349 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:53:47,349 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:53:47,349 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,350 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:53:47,352 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:47,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:47,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:47,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,354 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:47,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:47,357 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,359 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,359 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:47,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,361 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,361 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,361 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,361 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,361 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,361 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,361 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-27 03:53:47,361 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:47,361 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-27 03:53:47,361 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:53:47,361 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:47,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:47,362 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:47,363 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-27 03:53:47,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:47,363 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:53:47,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:47,463 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:53:47,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:53:47,480 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:53:47,494 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-27 03:53:47,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,495 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-27 03:53:47,495 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:47,495 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-27 03:53:47,496 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-27 03:53:47,496 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,496 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,498 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:47,498 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:47,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:47,498 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:47,498 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:53:47,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,499 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:53:47,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,499 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,500 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-27 03:53:47,500 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:47,500 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-27 03:53:47,500 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:47,501 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-27 03:53:47,501 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:53:47,501 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:53:47,501 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:47,501 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. started... 2023-03-27 03:53:47,502 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 647148480c2004d294d48558b731b404 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:53:47,507 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-27 03:53:47,508 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-03-27 03:53:47,586 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601010b7e0a47e4884a47ada7693524a0ba5b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601010b7e0a47e4884a47ada7693524a0ba5b 2023-03-27 03:53:48,030 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/19c60529ed6649e6aa35c16aa29dc1fb 2023-03-27 03:53:48,047 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/19c60529ed6649e6aa35c16aa29dc1fb as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/19c60529ed6649e6aa35c16aa29dc1fb 2023-03-27 03:53:48,062 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/19c60529ed6649e6aa35c16aa29dc1fb, entries=1, sequenceid=29, filesize=5.0 K 2023-03-27 03:53:48,065 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 647148480c2004d294d48558b731b404 in 564ms, sequenceid=29, compaction requested=true 2023-03-27 03:53:48,066 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 647148480c2004d294d48558b731b404: 2023-03-27 03:53:48,067 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:48,068 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:53:48,069 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:53:48,070 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,070 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-27 03:53:48,070 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-27 03:53:48,073 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,073 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:48,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:48,074 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,074 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:53:48,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:48,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:48,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:48,079 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-27 03:53:48,079 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6968cc30[Count = 0] remaining members to acquire global barrier 2023-03-27 03:53:48,079 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-27 03:53:48,079 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,081 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,082 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-27 03:53:48,082 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-27 03:53:48,082 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:53:48,082 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,082 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:53:48,084 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,085 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-27 03:53:48,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:48,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:48,085 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:48,085 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-27 03:53:48,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:48,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:48,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:48,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:53:48,089 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-27 03:53:48,090 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-27 03:53:48,090 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:53:48,090 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:53:48,090 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,090 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:53:48,092 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,092 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:48,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:48,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:48,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,094 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:48,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:48,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:48,096 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:48,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,100 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:48,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,108 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:48,108 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:48,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:48,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:48,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:48,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:48,109 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:48,109 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-27 03:53:48,109 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:53:48,110 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,110 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,110 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,110 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,110 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,110 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,110 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,111 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-27 03:53:48,111 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:53:48,211 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:53:48,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:53:48,226 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:53:48,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-27 03:53:48,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,241 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-27 03:53:48,241 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:48,242 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-27 03:53:48,242 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-27 03:53:48,243 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,243 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,244 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:48,244 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:48,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:48,244 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,244 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:53:48,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,245 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:53:48,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,246 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,252 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-27 03:53:48,252 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,252 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-27 03:53:48,252 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:48,261 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-27 03:53:48,264 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:53:48,264 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:53:48,264 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:48,264 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. started... 2023-03-27 03:53:48,264 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 647148480c2004d294d48558b731b404 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:53:48,319 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601030b44715d2ca342518c88056bd17550b9 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601030b44715d2ca342518c88056bd17550b9 2023-03-27 03:53:48,339 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/7978b234536e47aeb6681d1563f92de5 2023-03-27 03:53:48,360 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/7978b234536e47aeb6681d1563f92de5 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/7978b234536e47aeb6681d1563f92de5 2023-03-27 03:53:48,393 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/7978b234536e47aeb6681d1563f92de5, entries=1, sequenceid=33, filesize=5.0 K 2023-03-27 03:53:48,395 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 647148480c2004d294d48558b731b404 in 130ms, sequenceid=33, compaction requested=true 2023-03-27 03:53:48,395 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 647148480c2004d294d48558b731b404: 2023-03-27 03:53:48,395 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:48,395 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:53:48,396 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:53:48,396 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,396 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-27 03:53:48,396 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-27 03:53:48,398 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:48,398 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:48,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:48,399 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,400 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:53:48,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:48,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:48,402 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-27 03:53:48,402 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6610e94a[Count = 0] remaining members to acquire global barrier 2023-03-27 03:53:48,402 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-27 03:53:48,402 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,404 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,404 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-27 03:53:48,404 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-27 03:53:48,404 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:53:48,405 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,405 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:53:48,409 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:48,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:48,410 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-27 03:53:48,410 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:48,410 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-27 03:53:48,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:48,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:48,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:48,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:53:48,421 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-27 03:53:48,423 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-27 03:53:48,423 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:53:48,423 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:53:48,423 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,423 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:53:48,427 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:48,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:48,427 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:48,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,428 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:48,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:48,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,429 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:48,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:48,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,431 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:48,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,436 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,436 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,436 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,436 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,436 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,436 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,436 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,436 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:48,436 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:48,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:48,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:48,436 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:48,436 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-27 03:53:48,437 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:53:48,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:48,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:48,438 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-27 03:53:48,438 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:53:48,538 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:53:48,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:53:48,552 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:53:48,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-27 03:53:48,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,556 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-27 03:53:48,556 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:48,557 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-27 03:53:48,557 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-27 03:53:48,557 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,557 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,559 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:48,559 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:48,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:48,559 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:48,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,559 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:53:48,560 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:53:48,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,560 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,561 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-27 03:53:48,562 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:48,562 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-27 03:53:48,563 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:48,563 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-27 03:53:48,570 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:53:48,570 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:53:48,570 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:48,570 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. started... 2023-03-27 03:53:48,570 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 647148480c2004d294d48558b731b404 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:53:49,017 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509078b042c5df896466ca67ddd20fe1c7419 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201509078b042c5df896466ca67ddd20fe1c7419 2023-03-27 03:53:49,038 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/107421169f4948ad839d432b11e73ad2 2023-03-27 03:53:49,046 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/107421169f4948ad839d432b11e73ad2 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/107421169f4948ad839d432b11e73ad2 2023-03-27 03:53:49,053 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/107421169f4948ad839d432b11e73ad2, entries=1, sequenceid=37, filesize=5.0 K 2023-03-27 03:53:49,054 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 647148480c2004d294d48558b731b404 in 484ms, sequenceid=37, compaction requested=true 2023-03-27 03:53:49,054 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 647148480c2004d294d48558b731b404: 2023-03-27 03:53:49,054 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:49,054 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:53:49,055 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:53:49,055 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,055 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-27 03:53:49,055 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-27 03:53:49,058 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,058 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:49,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:49,058 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,058 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:53:49,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:49,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:49,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:49,062 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-27 03:53:49,062 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e6a2f0[Count = 0] remaining members to acquire global barrier 2023-03-27 03:53:49,062 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-27 03:53:49,062 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,063 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,063 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-27 03:53:49,063 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-27 03:53:49,063 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:53:49,063 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,063 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:53:49,065 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-27 03:53:49,065 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,065 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:49,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:49,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:49,065 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-27 03:53:49,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:49,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:49,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:49,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:53:49,069 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-27 03:53:49,069 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-27 03:53:49,070 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:53:49,070 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:53:49,070 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,070 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:53:49,071 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,071 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:49,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:49,071 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:49,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:49,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:49,072 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:49,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:49,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,074 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:49,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,079 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,080 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,080 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:49,080 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,080 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-27 03:53:49,080 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:53:49,080 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,080 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,080 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,080 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,080 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-27 03:53:49,081 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:53:49,081 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-27 03:53:49,082 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:49,082 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:49,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:49,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:49,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:49,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:49,181 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:53:49,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:53:49,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:53:49,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-03-27 03:53:49,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,213 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-27 03:53:49,213 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:49,220 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-03-27 03:53:49,220 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-03-27 03:53:49,223 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,223 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,233 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,233 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:49,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:49,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:49,233 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,233 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:53:49,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,235 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:53:49,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,235 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,236 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-03-27 03:53:49,236 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,236 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-03-27 03:53:49,236 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:49,237 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-03-27 03:53:49,238 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:53:49,239 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:53:49,239 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:49,239 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. started... 2023-03-27 03:53:49,239 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 647148480c2004d294d48558b731b404 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:53:49,283 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120ca0711875333453e8cac8fc4ff2afd49 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120ca0711875333453e8cac8fc4ff2afd49 2023-03-27 03:53:49,304 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/5a612c1594514b39ba0e18a0f9e553d0 2023-03-27 03:53:49,315 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/5a612c1594514b39ba0e18a0f9e553d0 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/5a612c1594514b39ba0e18a0f9e553d0 2023-03-27 03:53:49,326 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/5a612c1594514b39ba0e18a0f9e553d0, entries=1, sequenceid=41, filesize=5.0 K 2023-03-27 03:53:49,327 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 647148480c2004d294d48558b731b404 in 88ms, sequenceid=41, compaction requested=true 2023-03-27 03:53:49,327 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 647148480c2004d294d48558b731b404: 2023-03-27 03:53:49,327 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:49,328 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:53:49,328 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:53:49,328 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,328 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-03-27 03:53:49,328 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-03-27 03:53:49,330 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,330 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:49,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:49,331 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,331 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:53:49,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:49,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:49,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:49,336 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-03-27 03:53:49,337 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@52155dc3[Count = 0] remaining members to acquire global barrier 2023-03-27 03:53:49,337 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-03-27 03:53:49,337 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,338 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,338 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-03-27 03:53:49,338 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-03-27 03:53:49,338 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:53:49,339 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,340 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:53:49,345 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:49,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:49,346 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-03-27 03:53:49,346 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:49,346 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-03-27 03:53:49,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:49,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:49,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:49,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:53:49,354 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-03-27 03:53:49,354 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-03-27 03:53:49,354 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:53:49,355 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:53:49,355 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,355 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:53:49,356 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,356 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:49,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:49,357 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:49,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:49,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:49,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:49,360 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:49,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,361 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:49,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,364 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,364 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,364 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,364 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:49,364 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:49,364 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-03-27 03:53:49,364 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:49,364 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-03-27 03:53:49,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:49,364 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:49,364 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:53:49,364 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:49,364 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,364 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,365 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-03-27 03:53:49,365 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:53:49,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:49,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:49,465 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:53:49,467 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:53:49,473 DEBUG [Listener at localhost/36131] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-27 03:53:49,485 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34330, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-27 03:53:49,496 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38271] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:49,526 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-27 03:53:49,527 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-27 03:53:49,529 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679889829529 2023-03-27 03:53:49,530 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:53:49,532 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-27 03:53:49,539 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-27 03:53:49,539 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-27 03:53:49,540 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] zookeeper.ReadOnlyZKClient(139): Connect 0x2f8e5881 to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:53:49,589 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@59923290, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:53:49,600 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-27 03:53:50,317 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601034bc6d4e74f784eaea16f0d950b827cb2 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601034bc6d4e74f784eaea16f0d950b827cb2 2023-03-27 03:53:50,319 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-27 03:53:50,348 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-27 03:53:50,350 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60836, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-27 03:53:50,384 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-27 03:53:50,387 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34338, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-27 03:53:50,438 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/56b58bb6676149659e87cb876f0b25c9 first=Optional[r3] last=Optional[r6] 2023-03-27 03:53:50,465 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/56b58bb6676149659e87cb876f0b25c9}] 2023-03-27 03:53:50,500 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/56b58bb6676149659e87cb876f0b25c9 for inclusion in 647148480c2004d294d48558b731b404/family1 2023-03-27 03:53:50,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-27 03:53:50,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last= 2023-03-27 03:53:50,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for 647148480c2004d294d48558b731b404: 2023-03-27 03:53:50,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/56b58bb6676149659e87cb876f0b25c9 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ev4tfj73f16d72pmc4avqkf3u3gqarrnavlhd0baa5mtddjrc9ljmb8gnb0lo1ah/family1/56b58bb6676149659e87cb876f0b25c9 2023-03-27 03:53:50,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ev4tfj73f16d72pmc4avqkf3u3gqarrnavlhd0baa5mtddjrc9ljmb8gnb0lo1ah/family1/56b58bb6676149659e87cb876f0b25c9 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/b8544f58965d47048a63fdceee39f881_SeqId_44_ 2023-03-27 03:53:50,524 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ev4tfj73f16d72pmc4avqkf3u3gqarrnavlhd0baa5mtddjrc9ljmb8gnb0lo1ah/family1/56b58bb6676149659e87cb876f0b25c9 into 647148480c2004d294d48558b731b404/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/b8544f58965d47048a63fdceee39f881_SeqId_44_ - updating store file list. 2023-03-27 03:53:50,531 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/b8544f58965d47048a63fdceee39f881_SeqId_44_ into 647148480c2004d294d48558b731b404/family1 2023-03-27 03:53:50,531 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ev4tfj73f16d72pmc4avqkf3u3gqarrnavlhd0baa5mtddjrc9ljmb8gnb0lo1ah/family1/56b58bb6676149659e87cb876f0b25c9 into 647148480c2004d294d48558b731b404/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/b8544f58965d47048a63fdceee39f881_SeqId_44_) 2023-03-27 03:53:50,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ev4tfj73f16d72pmc4avqkf3u3gqarrnavlhd0baa5mtddjrc9ljmb8gnb0lo1ah/family1/56b58bb6676149659e87cb876f0b25c9 2023-03-27 03:53:50,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-27 03:53:50,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-27 03:53:50,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(1912): 647148480c2004d294d48558b731b404/family1 is initiating minor compaction 2023-03-27 03:53:50,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@729d13df; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:53:50,547 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 647148480c2004d294d48558b731b404/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:50,549 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/6f21901c2e534fc0b23391346b83006e, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/10dbc5a8cd9e471a83a58d7661ce493e, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/6da4899d3a4b44e49a3a88ea2f11ff41, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/eed40470ce27436ba67812f33572b49c, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/95f27c856fc14ce7b7c4a32a0ebe82fc, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/19c60529ed6649e6aa35c16aa29dc1fb, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/7978b234536e47aeb6681d1563f92de5, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/107421169f4948ad839d432b11e73ad2, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/5a612c1594514b39ba0e18a0f9e553d0, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/b8544f58965d47048a63fdceee39f881_SeqId_44_] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp, totalSize=50.2 K 2023-03-27 03:53:50,550 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 6f21901c2e534fc0b23391346b83006e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-27 03:53:50,550 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 10dbc5a8cd9e471a83a58d7661ce493e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-27 03:53:50,550 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 6da4899d3a4b44e49a3a88ea2f11ff41, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-27 03:53:50,551 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting eed40470ce27436ba67812f33572b49c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-27 03:53:50,551 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 95f27c856fc14ce7b7c4a32a0ebe82fc, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-27 03:53:50,551 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 19c60529ed6649e6aa35c16aa29dc1fb, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-27 03:53:50,551 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 7978b234536e47aeb6681d1563f92de5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-27 03:53:50,551 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 107421169f4948ad839d432b11e73ad2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-27 03:53:50,551 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 5a612c1594514b39ba0e18a0f9e553d0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-27 03:53:50,551 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting b8544f58965d47048a63fdceee39f881_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-27 03:53:50,579 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-27 03:53:50,582 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:53:50,588 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512283ce26f2981764e8caa21c7d2a830f5cb to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512283ce26f2981764e8caa21c7d2a830f5cb 2023-03-27 03:53:50,597 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512316e9e88ee92a14b30b09381e7bf49bd93 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512316e9e88ee92a14b30b09381e7bf49bd93 2023-03-27 03:53:50,603 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601010b7e0a47e4884a47ada7693524a0ba5b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601010b7e0a47e4884a47ada7693524a0ba5b 2023-03-27 03:53:50,605 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601030b44715d2ca342518c88056bd17550b9 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601030b44715d2ca342518c88056bd17550b9 2023-03-27 03:53:50,606 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-03-27 03:53:50,606 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-27 03:53:50,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-27 03:53:50,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-27 03:53:50,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-27 03:53:50,609 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-27 03:53:50,609 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-27 03:53:50,609 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-27 03:53:50,609 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-27 03:53:50,609 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-27 03:53:50,687 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] throttle.PressureAwareThroughputController(145): 647148480c2004d294d48558b731b404#family1#compaction#10 average throughput is 0.24 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:53:50,752 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/cd2eb45cecfa4f82978fb7efb09a4db0 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/cd2eb45cecfa4f82978fb7efb09a4db0 2023-03-27 03:53:50,795 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 647148480c2004d294d48558b731b404/family1 of 647148480c2004d294d48558b731b404 into cd2eb45cecfa4f82978fb7efb09a4db0(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:53:50,809 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/6f21901c2e534fc0b23391346b83006e, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/10dbc5a8cd9e471a83a58d7661ce493e, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/6da4899d3a4b44e49a3a88ea2f11ff41, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/eed40470ce27436ba67812f33572b49c, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/95f27c856fc14ce7b7c4a32a0ebe82fc, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/19c60529ed6649e6aa35c16aa29dc1fb, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/7978b234536e47aeb6681d1563f92de5, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/107421169f4948ad839d432b11e73ad2, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/5a612c1594514b39ba0e18a0f9e553d0, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/b8544f58965d47048a63fdceee39f881_SeqId_44_] to archive 2023-03-27 03:53:50,818 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:53:50,821 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/6f21901c2e534fc0b23391346b83006e to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/6f21901c2e534fc0b23391346b83006e 2023-03-27 03:53:50,823 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/10dbc5a8cd9e471a83a58d7661ce493e to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/10dbc5a8cd9e471a83a58d7661ce493e 2023-03-27 03:53:50,826 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/6da4899d3a4b44e49a3a88ea2f11ff41 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/6da4899d3a4b44e49a3a88ea2f11ff41 2023-03-27 03:53:50,835 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/eed40470ce27436ba67812f33572b49c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/eed40470ce27436ba67812f33572b49c 2023-03-27 03:53:50,839 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/95f27c856fc14ce7b7c4a32a0ebe82fc to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/95f27c856fc14ce7b7c4a32a0ebe82fc 2023-03-27 03:53:50,842 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 647148480c2004d294d48558b731b404: 2023-03-27 03:53:50,847 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1679889230544; duration=0sec 2023-03-27 03:53:50,850 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:53:50,850 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/19c60529ed6649e6aa35c16aa29dc1fb to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/19c60529ed6649e6aa35c16aa29dc1fb 2023-03-27 03:53:50,858 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/7978b234536e47aeb6681d1563f92de5 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/7978b234536e47aeb6681d1563f92de5 2023-03-27 03:53:50,861 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/107421169f4948ad839d432b11e73ad2 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/107421169f4948ad839d432b11e73ad2 2023-03-27 03:53:50,864 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/5a612c1594514b39ba0e18a0f9e553d0 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/5a612c1594514b39ba0e18a0f9e553d0 2023-03-27 03:53:50,867 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/b8544f58965d47048a63fdceee39f881_SeqId_44_ to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/b8544f58965d47048a63fdceee39f881_SeqId_44_ 2023-03-27 03:53:51,267 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512057d71e621b4ed4ccaa2dd993f6199eed3 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512057d71e621b4ed4ccaa2dd993f6199eed3 2023-03-27 03:53:51,269 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-27 03:53:51,341 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/94f53c330e8140d2abd15b67be9f0e24 first=Optional[r00] last=Optional[r2] 2023-03-27 03:53:51,343 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/94f53c330e8140d2abd15b67be9f0e24}] 2023-03-27 03:53:51,347 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/94f53c330e8140d2abd15b67be9f0e24 for inclusion in 647148480c2004d294d48558b731b404/family1 2023-03-27 03:53:51,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-27 03:53:51,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last= 2023-03-27 03:53:51,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for 647148480c2004d294d48558b731b404: 2023-03-27 03:53:51,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/94f53c330e8140d2abd15b67be9f0e24 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__p0cofpjjm6ig25smbhv9ruuf3ehbbrgvcjuta302s685ujmbbpde3s8ngpt4gk65/family1/94f53c330e8140d2abd15b67be9f0e24 2023-03-27 03:53:51,359 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__p0cofpjjm6ig25smbhv9ruuf3ehbbrgvcjuta302s685ujmbbpde3s8ngpt4gk65/family1/94f53c330e8140d2abd15b67be9f0e24 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/e69d8ea0e7b64e9083a84b33f016b9f8_SeqId_47_ 2023-03-27 03:53:51,360 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__p0cofpjjm6ig25smbhv9ruuf3ehbbrgvcjuta302s685ujmbbpde3s8ngpt4gk65/family1/94f53c330e8140d2abd15b67be9f0e24 into 647148480c2004d294d48558b731b404/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/e69d8ea0e7b64e9083a84b33f016b9f8_SeqId_47_ - updating store file list. 2023-03-27 03:53:51,369 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/e69d8ea0e7b64e9083a84b33f016b9f8_SeqId_47_ into 647148480c2004d294d48558b731b404/family1 2023-03-27 03:53:51,370 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__p0cofpjjm6ig25smbhv9ruuf3ehbbrgvcjuta302s685ujmbbpde3s8ngpt4gk65/family1/94f53c330e8140d2abd15b67be9f0e24 into 647148480c2004d294d48558b731b404/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/e69d8ea0e7b64e9083a84b33f016b9f8_SeqId_47_) 2023-03-27 03:53:51,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__p0cofpjjm6ig25smbhv9ruuf3ehbbrgvcjuta302s685ujmbbpde3s8ngpt4gk65/family1/94f53c330e8140d2abd15b67be9f0e24 2023-03-27 03:53:51,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-27 03:53:51,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-27 03:53:51,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(1912): 647148480c2004d294d48558b731b404/family1 is initiating minor compaction (all files) 2023-03-27 03:53:51,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@48288108; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:53:51,373 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 647148480c2004d294d48558b731b404/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:51,374 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/2ef373ea86eb4b0581d6e14e68800bf9, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/cd2eb45cecfa4f82978fb7efb09a4db0, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/e69d8ea0e7b64e9083a84b33f016b9f8_SeqId_47_] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp, totalSize=358.6 K 2023-03-27 03:53:51,375 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 2ef373ea86eb4b0581d6e14e68800bf9, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878842162 2023-03-27 03:53:51,378 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting cd2eb45cecfa4f82978fb7efb09a4db0, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621242162 2023-03-27 03:53:51,379 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting e69d8ea0e7b64e9083a84b33f016b9f8_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878842162 2023-03-27 03:53:51,380 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-27 03:53:51,383 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:53:51,386 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130395dd07d71db40279562deedffba0853 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130395dd07d71db40279562deedffba0853 2023-03-27 03:53:51,389 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512013864ea74254c40469996786d8f8e661b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512013864ea74254c40469996786d8f8e661b 2023-03-27 03:53:51,391 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512052a5ccb7fc1044ef4821b3c0e81b5c31e to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512052a5ccb7fc1044ef4821b3c0e81b5c31e 2023-03-27 03:53:51,391 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-03-27 03:53:51,391 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-27 03:53:51,393 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-27 03:53:51,393 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-27 03:53:51,394 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:53:51,396 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-27 03:53:51,400 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 1.8840 sec 2023-03-27 03:53:51,462 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] throttle.PressureAwareThroughputController(145): 647148480c2004d294d48558b731b404#family1#compaction#11 average throughput is 2.80 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:53:51,673 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-27 03:53:51,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:51,720 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-27 03:53:51,725 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-03-27 03:53:51,725 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-03-27 03:53:51,743 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-03-27 03:53:51,764 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=647148480c2004d294d48558b731b404, REOPEN/MOVE}] 2023-03-27 03:53:51,768 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=647148480c2004d294d48558b731b404, REOPEN/MOVE 2023-03-27 03:53:51,770 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=647148480c2004d294d48558b731b404, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:51,770 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679889231769"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889231769"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889231769"}]},"ts":"1679889231769"} 2023-03-27 03:53:51,775 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure 647148480c2004d294d48558b731b404, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:53:51,942 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 647148480c2004d294d48558b731b404 2023-03-27 03:53:51,942 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 647148480c2004d294d48558b731b404, disabling compactions & flushes 2023-03-27 03:53:51,942 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:52,342 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/.tmp/family1/75ce7e7a173449fe9e04e1fc29e7237e as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/75ce7e7a173449fe9e04e1fc29e7237e 2023-03-27 03:53:52,351 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 647148480c2004d294d48558b731b404/family1 of 647148480c2004d294d48558b731b404 into 75ce7e7a173449fe9e04e1fc29e7237e(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:53:52,351 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 647148480c2004d294d48558b731b404: 2023-03-27 03:53:52,351 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:52,351 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:52,351 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. after waiting 0 ms 2023-03-27 03:53:52,352 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:52,352 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1679889231372; duration=0sec 2023-03-27 03:53:52,352 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:53:52,359 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/2ef373ea86eb4b0581d6e14e68800bf9, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/cd2eb45cecfa4f82978fb7efb09a4db0, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/e69d8ea0e7b64e9083a84b33f016b9f8_SeqId_47_] to archive 2023-03-27 03:53:52,360 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:53:52,363 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/2ef373ea86eb4b0581d6e14e68800bf9 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/2ef373ea86eb4b0581d6e14e68800bf9 2023-03-27 03:53:52,372 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/cd2eb45cecfa4f82978fb7efb09a4db0 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/cd2eb45cecfa4f82978fb7efb09a4db0 2023-03-27 03:53:52,374 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/e69d8ea0e7b64e9083a84b33f016b9f8_SeqId_47_ to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/e69d8ea0e7b64e9083a84b33f016b9f8_SeqId_47_ 2023-03-27 03:53:52,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-27 03:53:52,398 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:52,398 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 647148480c2004d294d48558b731b404: 2023-03-27 03:53:52,398 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 647148480c2004d294d48558b731b404 to self. 2023-03-27 03:53:52,402 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 647148480c2004d294d48558b731b404 2023-03-27 03:53:52,402 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=647148480c2004d294d48558b731b404, regionState=CLOSED 2023-03-27 03:53:52,403 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679889232402"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889232402"}]},"ts":"1679889232402"} 2023-03-27 03:53:52,411 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-03-27 03:53:52,411 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure 647148480c2004d294d48558b731b404, server=jenkins-hbase4.apache.org,39727,1679889218560 in 630 msec 2023-03-27 03:53:52,412 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=647148480c2004d294d48558b731b404, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=true 2023-03-27 03:53:52,562 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=647148480c2004d294d48558b731b404, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:52,563 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679889232562"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889232562"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889232562"}]},"ts":"1679889232562"} 2023-03-27 03:53:52,566 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 647148480c2004d294d48558b731b404, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:53:52,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:52,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 647148480c2004d294d48558b731b404, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404.', STARTKEY => '', ENDKEY => ''} 2023-03-27 03:53:52,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 647148480c2004d294d48558b731b404 2023-03-27 03:53:52,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:53:52,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 647148480c2004d294d48558b731b404 2023-03-27 03:53:52,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 647148480c2004d294d48558b731b404 2023-03-27 03:53:52,727 INFO [StoreOpener-647148480c2004d294d48558b731b404-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 647148480c2004d294d48558b731b404 2023-03-27 03:53:52,728 DEBUG [StoreOpener-647148480c2004d294d48558b731b404-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1 2023-03-27 03:53:52,729 DEBUG [StoreOpener-647148480c2004d294d48558b731b404-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1 2023-03-27 03:53:52,729 INFO [StoreOpener-647148480c2004d294d48558b731b404-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 647148480c2004d294d48558b731b404 columnFamilyName family1 2023-03-27 03:53:52,740 DEBUG [StoreOpener-647148480c2004d294d48558b731b404-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/75ce7e7a173449fe9e04e1fc29e7237e 2023-03-27 03:53:52,741 INFO [StoreOpener-647148480c2004d294d48558b731b404-1] regionserver.HStore(310): Store=647148480c2004d294d48558b731b404/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:53:52,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404 2023-03-27 03:53:52,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404 2023-03-27 03:53:52,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 647148480c2004d294d48558b731b404 2023-03-27 03:53:52,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 647148480c2004d294d48558b731b404; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64348105, jitterRate=-0.04113851487636566}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-27 03:53:52,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 647148480c2004d294d48558b731b404: 2023-03-27 03:53:52,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404., pid=17, masterSystemTime=1679889232719 2023-03-27 03:53:52,757 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:52,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:53:52,757 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=647148480c2004d294d48558b731b404, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:52,757 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1679889232757"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889232757"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889232757"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889232757"}]},"ts":"1679889232757"} 2023-03-27 03:53:52,762 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-03-27 03:53:52,763 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 647148480c2004d294d48558b731b404, server=jenkins-hbase4.apache.org,39727,1679889218560 in 194 msec 2023-03-27 03:53:52,770 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-03-27 03:53:52,770 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=647148480c2004d294d48558b731b404, REOPEN/MOVE in 999 msec 2023-03-27 03:53:52,775 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-03-27 03:53:52,775 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.0300 sec 2023-03-27 03:53:52,778 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.1000 sec 2023-03-27 03:53:52,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-03-27 03:53:52,789 INFO [Listener at localhost/36131] compactions.TestMobCompactor(278): alter status finished 2023-03-27 03:53:52,790 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38271] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-03-27 03:53:52,793 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-27 03:53:52,794 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-27 03:53:52,795 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:53:52,795 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679889832795 2023-03-27 03:53:52,800 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-27 03:53:52,801 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-03-27 03:53:52,801 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-27 03:53:52,802 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] zookeeper.ReadOnlyZKClient(139): Connect 0x6db1089d to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:53:52,828 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@319eaec7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:53:52,829 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-27 03:53:52,829 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-27 03:53:52,829 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-27 03:53:52,829 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-27 03:53:52,829 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-27 03:53:52,916 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511289d0afb2410c442fd815892e3096fe010 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511289d0afb2410c442fd815892e3096fe010 2023-03-27 03:53:52,918 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-27 03:53:52,934 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-27 03:53:52,936 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60850, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-27 03:53:52,946 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-27 03:53:52,949 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34348, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-27 03:53:52,973 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/f6c7b4a2a6b54fdbbe4d53190855bb0c first=Optional[r05] last=Optional[r8] 2023-03-27 03:53:52,974 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=51 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/f6c7b4a2a6b54fdbbe4d53190855bb0c}] 2023-03-27 03:53:52,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/f6c7b4a2a6b54fdbbe4d53190855bb0c for inclusion in 647148480c2004d294d48558b731b404/family1 2023-03-27 03:53:52,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-27 03:53:52,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last= 2023-03-27 03:53:52,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for 647148480c2004d294d48558b731b404: 2023-03-27 03:53:52,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/f6c7b4a2a6b54fdbbe4d53190855bb0c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ch4084ob5p8vmn7ar01fjmirue0pjlem1jul936f1j7f6fqpn684t0isdkmh9cn8/family1/f6c7b4a2a6b54fdbbe4d53190855bb0c 2023-03-27 03:53:52,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ch4084ob5p8vmn7ar01fjmirue0pjlem1jul936f1j7f6fqpn684t0isdkmh9cn8/family1/f6c7b4a2a6b54fdbbe4d53190855bb0c as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/29246e63bf5a4f2ebcf91b773b4fb7e2_SeqId_53_ 2023-03-27 03:53:53,007 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ch4084ob5p8vmn7ar01fjmirue0pjlem1jul936f1j7f6fqpn684t0isdkmh9cn8/family1/f6c7b4a2a6b54fdbbe4d53190855bb0c into 647148480c2004d294d48558b731b404/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/29246e63bf5a4f2ebcf91b773b4fb7e2_SeqId_53_ - updating store file list. 2023-03-27 03:53:53,015 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/29246e63bf5a4f2ebcf91b773b4fb7e2_SeqId_53_ into 647148480c2004d294d48558b731b404/family1 2023-03-27 03:53:53,015 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ch4084ob5p8vmn7ar01fjmirue0pjlem1jul936f1j7f6fqpn684t0isdkmh9cn8/family1/f6c7b4a2a6b54fdbbe4d53190855bb0c into 647148480c2004d294d48558b731b404/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/family1/29246e63bf5a4f2ebcf91b773b4fb7e2_SeqId_53_) 2023-03-27 03:53:53,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ch4084ob5p8vmn7ar01fjmirue0pjlem1jul936f1j7f6fqpn684t0isdkmh9cn8/family1/f6c7b4a2a6b54fdbbe4d53190855bb0c 2023-03-27 03:53:53,021 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-27 03:53:53,022 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:53:53,025 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120ca0711875333453e8cac8fc4ff2afd49 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120ca0711875333453e8cac8fc4ff2afd49 2023-03-27 03:53:53,027 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128512816e53b204d04859e60220f63aa23 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128512816e53b204d04859e60220f63aa23 2023-03-27 03:53:53,027 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-03-27 03:53:53,027 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-27 03:53:53,028 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-27 03:53:53,028 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-27 03:53:53,028 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-03-27 03:53:53,028 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-03-27 03:53:53,029 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:53:53,030 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-03-27 03:53:53,033 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 238 msec 2023-03-27 03:53:53,189 INFO [Listener at localhost/36131] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=251 (was 212) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1709667654_15 at /127.0.0.1:41590 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1709667654_15 at /127.0.0.1:41616 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62246@0x6db1089d-SendThread(127.0.0.1:62246) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1709667654_15 at /127.0.0.1:41550 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1709667654_15 at /127.0.0.1:41520 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1709667654_15 at /127.0.0.1:41790 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1709667654_15 at /127.0.0.1:41738 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:39727-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1709667654_15 at /127.0.0.1:41760 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62246@0x2f8e5881-SendThread(127.0.0.1:62246) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: hconnection-0x5616fc7d-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x69792f4c-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1709667654_15 at /127.0.0.1:41834 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x69792f4c-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62246@0x6db1089d-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: hconnection-0x5616fc7d-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1709667654_15 at /127.0.0.1:41640 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62246@0x6db1089d sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1313415097.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2030476682_15 at /127.0.0.1:41608 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1709667654_15 at /127.0.0.1:41656 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62246@0x2f8e5881 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1313415097.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5616fc7d-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1709667654_15 at /127.0.0.1:41726 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1709667654_15 at /127.0.0.1:41688 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5616fc7d-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62246@0x2f8e5881-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1709667654_15 at /127.0.0.1:41488 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1709667654_15 at /127.0.0.1:41776 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x69792f4c-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1709667654_15 at /127.0.0.1:41628 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x5616fc7d-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=596 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=430 (was 381) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=9103 (was 9519) 2023-03-27 03:53:53,234 INFO [Listener at localhost/36131] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=251, OpenFileDescriptor=598, MaxFileDescriptor=60000, SystemLoadAverage=430, ProcessCount=170, AvailableMemoryMB=9102 2023-03-27 03:53:53,238 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-27 03:53:53,240 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:53,242 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-27 03:53:53,243 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-03-27 03:53:53,244 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-27 03:53:53,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-27 03:53:53,284 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3 2023-03-27 03:53:53,285 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3 empty. 2023-03-27 03:53:53,286 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3 2023-03-27 03:53:53,286 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-03-27 03:53:53,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-27 03:53:53,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-27 03:53:53,709 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-27 03:53:53,711 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 78fc96d1b1d88244802e871a9fbedaf3, NAME => 'testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp 2023-03-27 03:53:53,724 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:53:53,724 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 78fc96d1b1d88244802e871a9fbedaf3, disabling compactions & flushes 2023-03-27 03:53:53,724 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:53,724 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:53,724 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. after waiting 0 ms 2023-03-27 03:53:53,724 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:53,724 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:53,724 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 78fc96d1b1d88244802e871a9fbedaf3: 2023-03-27 03:53:53,727 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-27 03:53:53,728 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679889233728"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889233728"}]},"ts":"1679889233728"} 2023-03-27 03:53:53,730 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-27 03:53:53,731 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-27 03:53:53,731 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889233731"}]},"ts":"1679889233731"} 2023-03-27 03:53:53,733 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-27 03:53:53,741 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=78fc96d1b1d88244802e871a9fbedaf3, ASSIGN}] 2023-03-27 03:53:53,744 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=78fc96d1b1d88244802e871a9fbedaf3, ASSIGN 2023-03-27 03:53:53,745 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=78fc96d1b1d88244802e871a9fbedaf3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=false 2023-03-27 03:53:53,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-27 03:53:53,897 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=78fc96d1b1d88244802e871a9fbedaf3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:53,897 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679889233897"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889233897"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889233897"}]},"ts":"1679889233897"} 2023-03-27 03:53:53,907 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 78fc96d1b1d88244802e871a9fbedaf3, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:53:54,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:54,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 78fc96d1b1d88244802e871a9fbedaf3, NAME => 'testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3.', STARTKEY => '', ENDKEY => ''} 2023-03-27 03:53:54,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy 78fc96d1b1d88244802e871a9fbedaf3 2023-03-27 03:53:54,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:53:54,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 78fc96d1b1d88244802e871a9fbedaf3 2023-03-27 03:53:54,065 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 78fc96d1b1d88244802e871a9fbedaf3 2023-03-27 03:53:54,069 INFO [StoreOpener-78fc96d1b1d88244802e871a9fbedaf3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 78fc96d1b1d88244802e871a9fbedaf3 2023-03-27 03:53:54,072 DEBUG [StoreOpener-78fc96d1b1d88244802e871a9fbedaf3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1 2023-03-27 03:53:54,072 DEBUG [StoreOpener-78fc96d1b1d88244802e871a9fbedaf3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1 2023-03-27 03:53:54,073 INFO [StoreOpener-78fc96d1b1d88244802e871a9fbedaf3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 78fc96d1b1d88244802e871a9fbedaf3 columnFamilyName family1 2023-03-27 03:53:54,077 INFO [StoreOpener-78fc96d1b1d88244802e871a9fbedaf3-1] regionserver.HStore(310): Store=78fc96d1b1d88244802e871a9fbedaf3/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:53:54,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3 2023-03-27 03:53:54,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3 2023-03-27 03:53:54,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 78fc96d1b1d88244802e871a9fbedaf3 2023-03-27 03:53:54,083 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:53:54,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 78fc96d1b1d88244802e871a9fbedaf3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66398199, jitterRate=-0.010589733719825745}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-27 03:53:54,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 78fc96d1b1d88244802e871a9fbedaf3: 2023-03-27 03:53:54,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3., pid=21, masterSystemTime=1679889234060 2023-03-27 03:53:54,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:54,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:54,091 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=78fc96d1b1d88244802e871a9fbedaf3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:54,091 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679889234091"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889234091"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889234091"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889234091"}]},"ts":"1679889234091"} 2023-03-27 03:53:54,097 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-03-27 03:53:54,097 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 78fc96d1b1d88244802e871a9fbedaf3, server=jenkins-hbase4.apache.org,39727,1679889218560 in 187 msec 2023-03-27 03:53:54,100 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-03-27 03:53:54,100 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=78fc96d1b1d88244802e871a9fbedaf3, ASSIGN in 356 msec 2023-03-27 03:53:54,101 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-27 03:53:54,101 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889234101"}]},"ts":"1679889234101"} 2023-03-27 03:53:54,103 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-27 03:53:54,106 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-27 03:53:54,109 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 868 msec 2023-03-27 03:53:54,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-03-27 03:53:54,351 INFO [Listener at localhost/36131] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-03-27 03:53:54,405 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:53:54,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,419 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-27 03:53:54,419 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:54,420 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-27 03:53:54,420 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-27 03:53:54,421 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,421 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,422 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:54,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:54,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:54,423 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:54,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,423 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:54,423 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:53:54,424 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:53:54,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,425 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,425 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,425 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-27 03:53:54,425 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:54,426 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-27 03:53:54,426 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:53:54,426 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:53:54,426 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:54,426 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. started... 2023-03-27 03:53:54,427 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 78fc96d1b1d88244802e871a9fbedaf3 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-27 03:53:54,491 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130d0e1250ba7ac402286e5dd20119fb1ec to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130d0e1250ba7ac402286e5dd20119fb1ec 2023-03-27 03:53:54,499 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/e54d0cfe4f754feba36a53a23834c4db 2023-03-27 03:53:54,509 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/e54d0cfe4f754feba36a53a23834c4db as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/e54d0cfe4f754feba36a53a23834c4db 2023-03-27 03:53:54,518 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/e54d0cfe4f754feba36a53a23834c4db, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-27 03:53:54,520 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 78fc96d1b1d88244802e871a9fbedaf3 in 93ms, sequenceid=5, compaction requested=false 2023-03-27 03:53:54,520 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-03-27 03:53:54,522 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 78fc96d1b1d88244802e871a9fbedaf3: 2023-03-27 03:53:54,522 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:54,522 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:53:54,522 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:53:54,522 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:54,522 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-27 03:53:54,522 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-27 03:53:54,524 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:54,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:54,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:54,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:54,525 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:54,525 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,526 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:53:54,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:54,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:54,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:54,529 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-27 03:53:54,529 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7d3312fd[Count = 0] remaining members to acquire global barrier 2023-03-27 03:53:54,529 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-27 03:53:54,529 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,530 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,530 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-27 03:53:54,530 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-27 03:53:54,530 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:53:54,531 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:54,531 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:53:54,535 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-27 03:53:54,536 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:54,536 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-27 03:53:54,537 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:54,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:54,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:54,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:54,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:54,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:54,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:54,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:54,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:54,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:53:54,543 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:53:54,543 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-27 03:53:54,543 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:53:54,543 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:53:54,543 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,543 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:53:54,547 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,547 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:54,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:54,547 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:54,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:54,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:54,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:54,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,549 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:54,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:54,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:54,550 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:54,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:54,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:54,555 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:54,555 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:54,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:54,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:54,555 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:54,555 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,555 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,555 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:54,555 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,555 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,555 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-27 03:53:54,556 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:53:54,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:54,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:54,555 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,556 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:53:54,556 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:53:54,555 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:54,656 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:53:54,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:53:54,669 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:53:54,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:53:54,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,673 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-27 03:53:54,673 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:54,674 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-27 03:53:54,674 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-27 03:53:54,676 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,676 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,680 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:54,680 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:54,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:54,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:54,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,682 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,682 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:54,682 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:53:54,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:53:54,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,683 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,684 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-27 03:53:54,684 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:54,685 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-27 03:53:54,685 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:54,689 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-27 03:53:54,693 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:53:54,693 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:53:54,693 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:54,693 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. started... 2023-03-27 03:53:54,693 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 78fc96d1b1d88244802e871a9fbedaf3 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-27 03:53:54,756 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112804fc268eb5d64383ad02497bd50260eb to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112804fc268eb5d64383ad02497bd50260eb 2023-03-27 03:53:55,164 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/f0342138911442a58f70353e9ff8f0cf 2023-03-27 03:53:55,174 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/f0342138911442a58f70353e9ff8f0cf as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/f0342138911442a58f70353e9ff8f0cf 2023-03-27 03:53:55,187 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/f0342138911442a58f70353e9ff8f0cf, entries=1, sequenceid=9, filesize=5.0 K 2023-03-27 03:53:55,189 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 78fc96d1b1d88244802e871a9fbedaf3 in 496ms, sequenceid=9, compaction requested=false 2023-03-27 03:53:55,189 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 78fc96d1b1d88244802e871a9fbedaf3: 2023-03-27 03:53:55,189 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:55,189 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:53:55,189 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:53:55,189 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:55,189 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-27 03:53:55,189 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-27 03:53:55,191 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,191 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:55,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:55,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:55,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:55,192 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,192 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:53:55,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:55,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:55,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:55,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:55,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-27 03:53:55,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1c88fea8[Count = 0] remaining members to acquire global barrier 2023-03-27 03:53:55,195 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-27 03:53:55,195 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,196 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,196 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-27 03:53:55,196 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-27 03:53:55,196 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:53:55,198 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:55,198 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:53:55,200 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-27 03:53:55,200 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:55,200 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:55,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:55,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:55,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:55,200 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-27 03:53:55,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:55,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:55,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:55,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:55,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:55,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:53:55,204 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:53:55,204 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-27 03:53:55,204 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:53:55,204 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:53:55,204 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,204 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:53:55,206 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,206 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:55,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,206 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:55,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:55,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:55,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:55,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:55,208 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:55,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:55,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,210 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:55,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:55,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:55,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:55,215 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:55,215 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:55,215 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:55,215 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-27 03:53:55,215 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:53:55,215 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:55,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:55,215 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-27 03:53:55,215 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:55,215 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,216 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:55,216 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,217 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:55,217 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,217 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:53:55,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:55,217 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:53:55,317 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:53:55,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:53:55,327 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:53:55,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:53:55,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,340 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-27 03:53:55,340 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:55,341 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-27 03:53:55,341 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-27 03:53:55,342 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,342 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,343 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:55,343 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:55,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:55,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:55,343 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:55,343 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:53:55,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,344 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:53:55,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,345 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,345 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-27 03:53:55,345 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:55,345 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-27 03:53:55,345 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:55,346 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-27 03:53:55,346 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:53:55,346 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:53:55,346 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:55,346 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. started... 2023-03-27 03:53:55,346 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 78fc96d1b1d88244802e871a9fbedaf3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:53:55,765 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201caddeeeb22ba45a6baca24b46e78877b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201caddeeeb22ba45a6baca24b46e78877b 2023-03-27 03:53:56,175 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/c4c5ab5e50694399bb74238bd441632c 2023-03-27 03:53:56,196 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/c4c5ab5e50694399bb74238bd441632c as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/c4c5ab5e50694399bb74238bd441632c 2023-03-27 03:53:56,203 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/c4c5ab5e50694399bb74238bd441632c, entries=1, sequenceid=13, filesize=4.9 K 2023-03-27 03:53:56,205 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 78fc96d1b1d88244802e871a9fbedaf3 in 859ms, sequenceid=13, compaction requested=true 2023-03-27 03:53:56,205 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 78fc96d1b1d88244802e871a9fbedaf3: 2023-03-27 03:53:56,205 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:56,205 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:53:56,205 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:53:56,205 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,205 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-27 03:53:56,205 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-27 03:53:56,208 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:56,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:56,209 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:56,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:56,211 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,211 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:53:56,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:56,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-27 03:53:56,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2c493eeb[Count = 0] remaining members to acquire global barrier 2023-03-27 03:53:56,213 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-27 03:53:56,213 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,218 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,218 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-27 03:53:56,218 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-27 03:53:56,218 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:53:56,219 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,219 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:53:56,222 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-27 03:53:56,222 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:56,222 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-27 03:53:56,222 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:56,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:56,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:56,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:56,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:56,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:53:56,227 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:53:56,227 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-27 03:53:56,227 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:53:56,227 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:53:56,227 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,227 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:53:56,229 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,229 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:56,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:56,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:56,230 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:56,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:56,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,232 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:56,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:56,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,235 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:56,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,240 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,240 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,240 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,240 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,240 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,240 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,240 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,240 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:56,240 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:56,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:56,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:56,240 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-27 03:53:56,240 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-27 03:53:56,240 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:53:56,240 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:56,241 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:53:56,242 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:53:56,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:56,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:56,342 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:53:56,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:53:56,356 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:53:56,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:53:56,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,379 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-27 03:53:56,379 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:56,379 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-27 03:53:56,380 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-27 03:53:56,380 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,380 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,382 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:56,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:56,382 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:56,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,383 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:53:56,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,385 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,385 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:53:56,385 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,394 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-27 03:53:56,394 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,394 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-27 03:53:56,394 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:56,395 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-27 03:53:56,404 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:53:56,404 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:56,404 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:53:56,405 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. started... 2023-03-27 03:53:56,405 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 78fc96d1b1d88244802e871a9fbedaf3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:53:56,437 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205011003b6cae34138a2ee60f0fa2d97ab to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205011003b6cae34138a2ee60f0fa2d97ab 2023-03-27 03:53:56,448 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/9bfb49bf600b469485303f963ac55d5f 2023-03-27 03:53:56,457 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/9bfb49bf600b469485303f963ac55d5f as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/9bfb49bf600b469485303f963ac55d5f 2023-03-27 03:53:56,465 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/9bfb49bf600b469485303f963ac55d5f, entries=1, sequenceid=17, filesize=4.9 K 2023-03-27 03:53:56,466 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 78fc96d1b1d88244802e871a9fbedaf3 in 61ms, sequenceid=17, compaction requested=true 2023-03-27 03:53:56,466 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 78fc96d1b1d88244802e871a9fbedaf3: 2023-03-27 03:53:56,466 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:56,466 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:53:56,467 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:53:56,467 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,467 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-27 03:53:56,467 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-27 03:53:56,469 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,469 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:56,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:56,469 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,469 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:53:56,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:56,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:56,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:56,471 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-27 03:53:56,471 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@544a95e4[Count = 0] remaining members to acquire global barrier 2023-03-27 03:53:56,471 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-27 03:53:56,471 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,473 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,473 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:53:56,479 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,480 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-27 03:53:56,480 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-27 03:53:56,480 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:53:56,482 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-27 03:53:56,482 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:56,482 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-27 03:53:56,483 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:56,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:56,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:56,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:56,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:56,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:53:56,487 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:53:56,487 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-27 03:53:56,487 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:53:56,487 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:53:56,487 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,487 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:53:56,489 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:56,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:56,490 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,490 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:56,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:56,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:56,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:56,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,491 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:56,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:56,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,496 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,500 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:56,500 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:56,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:56,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:56,501 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,501 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,501 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,501 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:56,501 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-27 03:53:56,501 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:53:56,501 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,502 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,502 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,502 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:56,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:56,502 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:53:56,502 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:53:56,602 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:53:56,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:53:56,612 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:53:56,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:53:56,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,615 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-27 03:53:56,615 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:56,615 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-27 03:53:56,615 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-27 03:53:56,616 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,616 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,618 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,618 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:56,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:56,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:56,618 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:56,618 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:53:56,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,619 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:53:56,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,619 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,619 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-27 03:53:56,619 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:56,619 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-27 03:53:56,619 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:56,620 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-27 03:53:56,620 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:53:56,620 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:53:56,620 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:56,620 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. started... 2023-03-27 03:53:56,621 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 78fc96d1b1d88244802e871a9fbedaf3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:53:57,041 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228ca3f38e0198a42c2902d9aa4c4b4a205 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228ca3f38e0198a42c2902d9aa4c4b4a205 2023-03-27 03:53:57,450 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/d7b466865d6b4ee59ae030d50e592ccd 2023-03-27 03:53:57,457 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/d7b466865d6b4ee59ae030d50e592ccd as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/d7b466865d6b4ee59ae030d50e592ccd 2023-03-27 03:53:57,464 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/d7b466865d6b4ee59ae030d50e592ccd, entries=1, sequenceid=21, filesize=4.9 K 2023-03-27 03:53:57,465 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 78fc96d1b1d88244802e871a9fbedaf3 in 844ms, sequenceid=21, compaction requested=true 2023-03-27 03:53:57,465 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 78fc96d1b1d88244802e871a9fbedaf3: 2023-03-27 03:53:57,465 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:57,465 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:53:57,465 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:53:57,465 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,465 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-27 03:53:57,465 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-27 03:53:57,467 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,467 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:57,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:57,468 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,468 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:53:57,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:57,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:57,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:57,470 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-27 03:53:57,470 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@310bc16a[Count = 0] remaining members to acquire global barrier 2023-03-27 03:53:57,470 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-27 03:53:57,470 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,471 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,471 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-27 03:53:57,472 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-27 03:53:57,472 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:53:57,472 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,472 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:53:57,478 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,478 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-27 03:53:57,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:57,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:57,478 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:57,479 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-27 03:53:57,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:57,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:57,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:57,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:53:57,487 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:53:57,487 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-27 03:53:57,487 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:53:57,487 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:53:57,487 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,487 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:53:57,492 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,492 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:57,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,492 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:57,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:57,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:57,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:57,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:57,495 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:57,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,497 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:57,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,500 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:57,500 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:57,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:57,500 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-27 03:53:57,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:57,500 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-27 03:53:57,500 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:53:57,500 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,500 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,501 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,501 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,501 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,501 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,501 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,500 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:57,501 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:53:57,516 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:53:57,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:57,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:57,616 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:53:57,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:53:57,624 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:53:57,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:53:57,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,626 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-27 03:53:57,627 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:57,627 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-27 03:53:57,627 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-27 03:53:57,628 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,628 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,629 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:57,629 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:57,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:57,630 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,630 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:53:57,630 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:53:57,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,630 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,631 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-27 03:53:57,631 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,631 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-27 03:53:57,631 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:57,631 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-27 03:53:57,632 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:53:57,632 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:53:57,632 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:57,632 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. started... 2023-03-27 03:53:57,632 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 78fc96d1b1d88244802e871a9fbedaf3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:53:57,650 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231c338f2f166e0416d8082de42fc963661 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231c338f2f166e0416d8082de42fc963661 2023-03-27 03:53:57,657 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/7f0cefa5da584ca499676f67bddafc31 2023-03-27 03:53:57,664 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/7f0cefa5da584ca499676f67bddafc31 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/7f0cefa5da584ca499676f67bddafc31 2023-03-27 03:53:57,671 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/7f0cefa5da584ca499676f67bddafc31, entries=1, sequenceid=25, filesize=4.9 K 2023-03-27 03:53:57,672 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 78fc96d1b1d88244802e871a9fbedaf3 in 40ms, sequenceid=25, compaction requested=true 2023-03-27 03:53:57,672 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 78fc96d1b1d88244802e871a9fbedaf3: 2023-03-27 03:53:57,672 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:57,672 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:53:57,672 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:53:57,672 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,672 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-27 03:53:57,672 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-27 03:53:57,674 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,674 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:57,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:57,675 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,675 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:53:57,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:57,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:57,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:57,676 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-27 03:53:57,677 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6aad145d[Count = 0] remaining members to acquire global barrier 2023-03-27 03:53:57,677 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-27 03:53:57,677 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,679 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,679 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-27 03:53:57,679 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-27 03:53:57,679 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:53:57,679 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,679 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:53:57,680 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:57,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:57,681 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-27 03:53:57,681 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:57,681 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-27 03:53:57,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:57,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:53:57,684 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:53:57,684 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-27 03:53:57,684 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:53:57,684 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:53:57,684 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,684 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:53:57,686 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,686 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:57,686 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:57,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:57,687 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:57,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:57,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,688 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:57,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,690 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,690 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:57,690 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,691 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,691 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:57,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:57,691 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:57,691 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,691 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-27 03:53:57,691 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-27 03:53:57,691 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:53:57,691 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,691 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:57,691 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:57,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:57,692 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:53:57,692 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:53:57,792 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:53:57,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:53:57,800 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:53:57,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:53:57,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,802 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-27 03:53:57,802 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:57,802 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-27 03:53:57,802 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-27 03:53:57,803 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,803 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,805 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:57,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:57,805 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:57,805 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,806 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:53:57,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,806 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:53:57,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,806 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,807 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-27 03:53:57,807 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,807 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-27 03:53:57,807 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:57,807 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-27 03:53:57,808 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:53:57,808 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:53:57,808 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:57,808 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. started... 2023-03-27 03:53:57,808 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 78fc96d1b1d88244802e871a9fbedaf3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:53:57,839 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101843643ec16c54d9b9df79b1f7ee0c46a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101843643ec16c54d9b9df79b1f7ee0c46a 2023-03-27 03:53:57,845 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/8ae54e9139644737992ba8e9ad4d0212 2023-03-27 03:53:57,852 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/8ae54e9139644737992ba8e9ad4d0212 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/8ae54e9139644737992ba8e9ad4d0212 2023-03-27 03:53:57,858 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/8ae54e9139644737992ba8e9ad4d0212, entries=1, sequenceid=29, filesize=4.9 K 2023-03-27 03:53:57,860 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 78fc96d1b1d88244802e871a9fbedaf3 in 52ms, sequenceid=29, compaction requested=true 2023-03-27 03:53:57,860 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 78fc96d1b1d88244802e871a9fbedaf3: 2023-03-27 03:53:57,860 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:57,860 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:53:57,860 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:53:57,860 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,860 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-27 03:53:57,860 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-27 03:53:57,862 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,862 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:57,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:57,862 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,862 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:53:57,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:57,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:57,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:57,864 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-27 03:53:57,864 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ff6c185[Count = 0] remaining members to acquire global barrier 2023-03-27 03:53:57,864 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-27 03:53:57,864 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,865 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,865 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-27 03:53:57,865 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-27 03:53:57,865 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:53:57,865 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,865 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:53:57,867 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,867 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-27 03:53:57,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:57,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:57,867 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:57,867 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-27 03:53:57,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:57,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:57,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:57,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:53:57,871 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:53:57,871 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-27 03:53:57,871 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:53:57,871 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:53:57,871 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,871 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:53:57,874 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,874 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,874 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:57,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:57,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:57,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:57,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:57,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:57,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:57,876 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:57,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,877 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,880 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,880 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:57,880 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,880 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:57,881 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,880 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-27 03:53:57,881 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:53:57,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:57,880 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:57,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:57,881 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:57,881 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,882 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,882 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,882 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:53:57,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:57,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:57,882 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:53:57,943 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-27 03:53:57,982 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:53:57,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:53:57,993 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:53:57,997 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:53:57,997 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:57,997 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-27 03:53:57,997 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:57,998 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-27 03:53:57,998 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-27 03:53:58,001 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,001 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,003 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:58,003 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:58,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:58,003 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,003 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:53:58,004 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:53:58,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,005 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,005 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-27 03:53:58,005 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,005 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-27 03:53:58,005 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:58,006 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-27 03:53:58,006 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:53:58,006 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:53:58,006 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:58,006 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. started... 2023-03-27 03:53:58,006 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 78fc96d1b1d88244802e871a9fbedaf3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:53:58,034 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010366defcce98ef4fd18d97cbfaad3b6670 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010366defcce98ef4fd18d97cbfaad3b6670 2023-03-27 03:53:58,057 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/24ed271a141d42148f4ea5ae25bd0ec9 2023-03-27 03:53:58,066 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/24ed271a141d42148f4ea5ae25bd0ec9 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/24ed271a141d42148f4ea5ae25bd0ec9 2023-03-27 03:53:58,073 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/24ed271a141d42148f4ea5ae25bd0ec9, entries=1, sequenceid=33, filesize=4.9 K 2023-03-27 03:53:58,076 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 78fc96d1b1d88244802e871a9fbedaf3 in 70ms, sequenceid=33, compaction requested=true 2023-03-27 03:53:58,076 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 78fc96d1b1d88244802e871a9fbedaf3: 2023-03-27 03:53:58,076 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:58,076 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:53:58,076 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:53:58,076 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,076 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-27 03:53:58,076 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-27 03:53:58,078 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,078 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:58,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:58,078 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,078 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:53:58,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:58,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:58,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:58,080 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-27 03:53:58,080 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@f13802a[Count = 0] remaining members to acquire global barrier 2023-03-27 03:53:58,080 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-27 03:53:58,080 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,082 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,083 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-27 03:53:58,083 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-27 03:53:58,083 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:53:58,083 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,083 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:53:58,086 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-27 03:53:58,086 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:58,086 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,087 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-27 03:53:58,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:58,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:58,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:58,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:58,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:58,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:53:58,090 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:53:58,090 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-27 03:53:58,090 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:53:58,090 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:53:58,090 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,090 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:53:58,092 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,092 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:58,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:58,092 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:58,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:58,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:58,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:58,093 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:58,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,094 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:58,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,098 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:58,098 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:58,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:58,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:58,098 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:58,098 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,099 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,099 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,099 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,099 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,099 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,099 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,098 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-27 03:53:58,099 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:53:58,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:58,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:58,099 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-27 03:53:58,100 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:53:58,100 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:53:58,200 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:53:58,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:53:58,209 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:53:58,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:53:58,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,213 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-27 03:53:58,213 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:58,214 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-27 03:53:58,214 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-27 03:53:58,216 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,216 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,226 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:58,226 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:58,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:58,227 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,227 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:53:58,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,227 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:53:58,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,228 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,228 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-27 03:53:58,228 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,228 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-27 03:53:58,228 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:58,229 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-27 03:53:58,236 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:53:58,236 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:53:58,236 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:58,236 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. started... 2023-03-27 03:53:58,236 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 78fc96d1b1d88244802e871a9fbedaf3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:53:58,664 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907a20710d2c1c84081a19bb0ec50e45947 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20150907a20710d2c1c84081a19bb0ec50e45947 2023-03-27 03:53:58,672 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/220a269f1d56425a821fb7180bfbc712 2023-03-27 03:53:58,679 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/220a269f1d56425a821fb7180bfbc712 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/220a269f1d56425a821fb7180bfbc712 2023-03-27 03:53:58,686 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/220a269f1d56425a821fb7180bfbc712, entries=1, sequenceid=37, filesize=4.9 K 2023-03-27 03:53:58,687 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 78fc96d1b1d88244802e871a9fbedaf3 in 451ms, sequenceid=37, compaction requested=true 2023-03-27 03:53:58,687 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 78fc96d1b1d88244802e871a9fbedaf3: 2023-03-27 03:53:58,687 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:58,687 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:53:58,687 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:53:58,687 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,687 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-27 03:53:58,688 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-27 03:53:58,689 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,689 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:58,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:58,689 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,690 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:53:58,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:58,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:58,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:58,691 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-27 03:53:58,691 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60fc0033[Count = 0] remaining members to acquire global barrier 2023-03-27 03:53:58,691 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-27 03:53:58,691 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,692 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,693 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-27 03:53:58,693 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,693 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:53:58,693 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-27 03:53:58,693 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:53:58,695 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,695 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-27 03:53:58,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:58,696 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:58,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:58,696 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-27 03:53:58,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:58,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:58,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:58,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:53:58,699 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:53:58,699 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-27 03:53:58,699 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:53:58,699 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:53:58,699 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,699 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:53:58,701 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,701 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:58,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:58,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,701 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:58,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,702 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:58,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:58,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:58,704 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:58,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:58,707 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,707 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,707 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:58,707 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,707 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:58,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:58,707 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-27 03:53:58,708 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:53:58,707 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,707 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:58,708 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,708 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,708 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:58,708 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:53:58,708 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:53:58,708 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-27 03:53:58,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:58,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:58,808 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:53:58,810 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:53:58,816 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:53:58,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:53:58,818 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,818 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-03-27 03:53:58,818 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:58,818 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-27 03:53:58,818 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-27 03:53:58,819 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,819 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,820 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,820 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:58,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:58,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:58,821 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,821 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:53:58,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,822 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:53:58,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,822 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,823 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-03-27 03:53:58,823 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,823 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-27 03:53:58,823 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:53:58,825 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-27 03:53:58,825 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:53:58,825 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:58,825 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:53:58,825 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. started... 2023-03-27 03:53:58,825 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 78fc96d1b1d88244802e871a9fbedaf3 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:53:58,843 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120792552ec7ab948e0823b06728f9f145a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120792552ec7ab948e0823b06728f9f145a 2023-03-27 03:53:58,856 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/43dc72ce53f046d3814e5368655de6dd 2023-03-27 03:53:58,863 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/43dc72ce53f046d3814e5368655de6dd as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/43dc72ce53f046d3814e5368655de6dd 2023-03-27 03:53:58,868 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/43dc72ce53f046d3814e5368655de6dd, entries=1, sequenceid=41, filesize=4.9 K 2023-03-27 03:53:58,869 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 78fc96d1b1d88244802e871a9fbedaf3 in 44ms, sequenceid=41, compaction requested=true 2023-03-27 03:53:58,870 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 78fc96d1b1d88244802e871a9fbedaf3: 2023-03-27 03:53:58,870 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:58,870 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:53:58,870 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:53:58,870 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,870 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-03-27 03:53:58,870 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-03-27 03:53:58,873 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,873 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:58,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:58,874 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,874 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:53:58,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:58,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:58,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:58,875 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-03-27 03:53:58,875 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62f1ff8d[Count = 0] remaining members to acquire global barrier 2023-03-27 03:53:58,875 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-27 03:53:58,875 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,876 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,877 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-27 03:53:58,877 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-03-27 03:53:58,877 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:53:58,877 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,877 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:53:58,878 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,878 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-27 03:53:58,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:58,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:58,878 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:58,879 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-03-27 03:53:58,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:58,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:58,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:58,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:53:58,882 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:53:58,882 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-03-27 03:53:58,882 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:53:58,882 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:53:58,882 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,882 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:53:58,884 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,884 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:53:58,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:53:58,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,884 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:58,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:53:58,884 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:58,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:58,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:53:58,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,886 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:53:58,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,901 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:53:58,901 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:53:58,901 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,901 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:53:58,902 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-03-27 03:53:58,903 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:53:58,904 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:53:58,904 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:53:58,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:53:58,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:53:58,901 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,905 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,905 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:53:58,905 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,905 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,905 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:58,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:53:58,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:53:59,004 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:53:59,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:53:59,007 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38271] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-03-27 03:53:59,012 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-27 03:53:59,013 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-27 03:53:59,020 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:53:59,021 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-27 03:53:59,023 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-27 03:53:59,023 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-27 03:53:59,023 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] zookeeper.ReadOnlyZKClient(139): Connect 0x2f4dd525 to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:53:59,020 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679889839020 2023-03-27 03:53:59,030 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6b61ae9d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:53:59,030 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-27 03:53:59,884 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103372c5d5781904c6cbee5248246963970 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103372c5d5781904c6cbee5248246963970 2023-03-27 03:53:59,890 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-27 03:53:59,908 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-27 03:53:59,911 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49012, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-27 03:53:59,922 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-27 03:53:59,923 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33310, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-27 03:53:59,962 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/1be8aab6838348749dcf7834078e4a2f first=Optional[r5] last=Optional[r6] 2023-03-27 03:53:59,963 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/1be8aab6838348749dcf7834078e4a2f}] 2023-03-27 03:53:59,970 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/1be8aab6838348749dcf7834078e4a2f for inclusion in 78fc96d1b1d88244802e871a9fbedaf3/family1 2023-03-27 03:53:59,974 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-27 03:53:59,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last= 2023-03-27 03:53:59,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for 78fc96d1b1d88244802e871a9fbedaf3: 2023-03-27 03:53:59,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/1be8aab6838348749dcf7834078e4a2f to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithMonthlyPolicy__s7pi1gs8crh2dd49ranv2anrk0oucoiprifrrhjteot0p9v67hk2m76c83pn5ln3/family1/1be8aab6838348749dcf7834078e4a2f 2023-03-27 03:53:59,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithMonthlyPolicy__s7pi1gs8crh2dd49ranv2anrk0oucoiprifrrhjteot0p9v67hk2m76c83pn5ln3/family1/1be8aab6838348749dcf7834078e4a2f as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/78b772629d6b46c8b87cf4f7e0bca276_SeqId_44_ 2023-03-27 03:53:59,983 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithMonthlyPolicy__s7pi1gs8crh2dd49ranv2anrk0oucoiprifrrhjteot0p9v67hk2m76c83pn5ln3/family1/1be8aab6838348749dcf7834078e4a2f into 78fc96d1b1d88244802e871a9fbedaf3/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/78b772629d6b46c8b87cf4f7e0bca276_SeqId_44_ - updating store file list. 2023-03-27 03:53:59,990 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/78b772629d6b46c8b87cf4f7e0bca276_SeqId_44_ into 78fc96d1b1d88244802e871a9fbedaf3/family1 2023-03-27 03:53:59,990 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithMonthlyPolicy__s7pi1gs8crh2dd49ranv2anrk0oucoiprifrrhjteot0p9v67hk2m76c83pn5ln3/family1/1be8aab6838348749dcf7834078e4a2f into 78fc96d1b1d88244802e871a9fbedaf3/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/78b772629d6b46c8b87cf4f7e0bca276_SeqId_44_) 2023-03-27 03:53:59,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithMonthlyPolicy__s7pi1gs8crh2dd49ranv2anrk0oucoiprifrrhjteot0p9v67hk2m76c83pn5ln3/family1/1be8aab6838348749dcf7834078e4a2f 2023-03-27 03:53:59,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-27 03:53:59,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-27 03:53:59,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(1912): 78fc96d1b1d88244802e871a9fbedaf3/family1 is initiating minor compaction 2023-03-27 03:53:59,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@372e0e7e; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:53:59,995 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 78fc96d1b1d88244802e871a9fbedaf3/family1 in testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:53:59,995 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/f0342138911442a58f70353e9ff8f0cf, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/c4c5ab5e50694399bb74238bd441632c, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/9bfb49bf600b469485303f963ac55d5f, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/d7b466865d6b4ee59ae030d50e592ccd, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/7f0cefa5da584ca499676f67bddafc31, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/8ae54e9139644737992ba8e9ad4d0212, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/24ed271a141d42148f4ea5ae25bd0ec9, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/220a269f1d56425a821fb7180bfbc712, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/43dc72ce53f046d3814e5368655de6dd, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/78b772629d6b46c8b87cf4f7e0bca276_SeqId_44_] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp, totalSize=49.7 K 2023-03-27 03:53:59,995 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting f0342138911442a58f70353e9ff8f0cf, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-27 03:53:59,996 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting c4c5ab5e50694399bb74238bd441632c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-27 03:53:59,996 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 9bfb49bf600b469485303f963ac55d5f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-27 03:53:59,996 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting d7b466865d6b4ee59ae030d50e592ccd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-27 03:53:59,996 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 7f0cefa5da584ca499676f67bddafc31, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-27 03:53:59,996 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 8ae54e9139644737992ba8e9ad4d0212, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-27 03:53:59,996 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 24ed271a141d42148f4ea5ae25bd0ec9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-27 03:53:59,996 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 220a269f1d56425a821fb7180bfbc712, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-27 03:53:59,996 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 43dc72ce53f046d3814e5368655de6dd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-27 03:53:59,996 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 78b772629d6b46c8b87cf4f7e0bca276_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-27 03:53:59,998 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-27 03:53:59,999 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:00,001 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101843643ec16c54d9b9df79b1f7ee0c46a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101843643ec16c54d9b9df79b1f7ee0c46a 2023-03-27 03:54:00,003 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010366defcce98ef4fd18d97cbfaad3b6670 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010366defcce98ef4fd18d97cbfaad3b6670 2023-03-27 03:54:00,003 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-27 03:54:00,003 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-27 03:54:00,003 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-27 03:54:00,003 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-27 03:54:00,104 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] throttle.PressureAwareThroughputController(145): 78fc96d1b1d88244802e871a9fbedaf3#family1#compaction#22 average throughput is 0.11 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:54:00,584 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-27 03:54:00,585 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6db1089d to 127.0.0.1:62246 2023-03-27 03:54:00,585 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:54:00,589 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-27 03:54:00,589 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2f8e5881 to 127.0.0.1:62246 2023-03-27 03:54:00,589 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:54:00,598 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/9722413088ec482fba0d297ead9903e7 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/9722413088ec482fba0d297ead9903e7 2023-03-27 03:54:00,599 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231970417d0e214498e8cad08a84f3a8ed7 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231970417d0e214498e8cad08a84f3a8ed7 2023-03-27 03:54:00,601 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-27 03:54:00,609 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 78fc96d1b1d88244802e871a9fbedaf3/family1 of 78fc96d1b1d88244802e871a9fbedaf3 into 9722413088ec482fba0d297ead9903e7(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:54:00,609 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 78fc96d1b1d88244802e871a9fbedaf3: 2023-03-27 03:54:00,609 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1679889239995; duration=0sec 2023-03-27 03:54:00,609 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:00,644 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/fe595e1e5de9446998fe3949fa8a7670 first=Optional[r1] last=Optional[r4] 2023-03-27 03:54:00,645 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/fe595e1e5de9446998fe3949fa8a7670}] 2023-03-27 03:54:00,649 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/fe595e1e5de9446998fe3949fa8a7670 for inclusion in 78fc96d1b1d88244802e871a9fbedaf3/family1 2023-03-27 03:54:00,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-27 03:54:00,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last= 2023-03-27 03:54:00,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for 78fc96d1b1d88244802e871a9fbedaf3: 2023-03-27 03:54:00,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/fe595e1e5de9446998fe3949fa8a7670 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithMonthlyPolicy__q3snnq64ppmk2n14a5d0u0ra58nt17ngj90lsost91emmscfndi25kla07hqf2pl/family1/fe595e1e5de9446998fe3949fa8a7670 2023-03-27 03:54:00,657 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithMonthlyPolicy__q3snnq64ppmk2n14a5d0u0ra58nt17ngj90lsost91emmscfndi25kla07hqf2pl/family1/fe595e1e5de9446998fe3949fa8a7670 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/ab156cdb3d14444fb159802fa64ebb1c_SeqId_47_ 2023-03-27 03:54:00,658 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithMonthlyPolicy__q3snnq64ppmk2n14a5d0u0ra58nt17ngj90lsost91emmscfndi25kla07hqf2pl/family1/fe595e1e5de9446998fe3949fa8a7670 into 78fc96d1b1d88244802e871a9fbedaf3/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/ab156cdb3d14444fb159802fa64ebb1c_SeqId_47_ - updating store file list. 2023-03-27 03:54:00,663 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/ab156cdb3d14444fb159802fa64ebb1c_SeqId_47_ into 78fc96d1b1d88244802e871a9fbedaf3/family1 2023-03-27 03:54:00,663 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithMonthlyPolicy__q3snnq64ppmk2n14a5d0u0ra58nt17ngj90lsost91emmscfndi25kla07hqf2pl/family1/fe595e1e5de9446998fe3949fa8a7670 into 78fc96d1b1d88244802e871a9fbedaf3/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/ab156cdb3d14444fb159802fa64ebb1c_SeqId_47_) 2023-03-27 03:54:00,663 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithMonthlyPolicy__q3snnq64ppmk2n14a5d0u0ra58nt17ngj90lsost91emmscfndi25kla07hqf2pl/family1/fe595e1e5de9446998fe3949fa8a7670 2023-03-27 03:54:00,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-27 03:54:00,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-27 03:54:00,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(1912): 78fc96d1b1d88244802e871a9fbedaf3/family1 is initiating minor compaction (all files) 2023-03-27 03:54:00,665 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 78fc96d1b1d88244802e871a9fbedaf3/family1 in testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:54:00,665 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/e54d0cfe4f754feba36a53a23834c4db, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/9722413088ec482fba0d297ead9903e7, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/ab156cdb3d14444fb159802fa64ebb1c_SeqId_47_] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp, totalSize=178.1 K 2023-03-27 03:54:00,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@61d79976; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:00,666 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting e54d0cfe4f754feba36a53a23834c4db, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878842162 2023-03-27 03:54:00,666 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 9722413088ec482fba0d297ead9903e7, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621242162 2023-03-27 03:54:00,667 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting ab156cdb3d14444fb159802fa64ebb1c_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965242162 2023-03-27 03:54:00,668 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-27 03:54:00,669 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:00,673 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201caddeeeb22ba45a6baca24b46e78877b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201caddeeeb22ba45a6baca24b46e78877b 2023-03-27 03:54:00,675 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205011003b6cae34138a2ee60f0fa2d97ab to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205011003b6cae34138a2ee60f0fa2d97ab 2023-03-27 03:54:00,676 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228ca3f38e0198a42c2902d9aa4c4b4a205 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228ca3f38e0198a42c2902d9aa4c4b4a205 2023-03-27 03:54:00,678 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231c338f2f166e0416d8082de42fc963661 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231c338f2f166e0416d8082de42fc963661 2023-03-27 03:54:00,678 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-27 03:54:00,678 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-27 03:54:00,679 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-27 03:54:00,679 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-27 03:54:00,734 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/f0342138911442a58f70353e9ff8f0cf, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/c4c5ab5e50694399bb74238bd441632c, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/9bfb49bf600b469485303f963ac55d5f, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/d7b466865d6b4ee59ae030d50e592ccd, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/7f0cefa5da584ca499676f67bddafc31, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/8ae54e9139644737992ba8e9ad4d0212, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/24ed271a141d42148f4ea5ae25bd0ec9, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/220a269f1d56425a821fb7180bfbc712, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/43dc72ce53f046d3814e5368655de6dd, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/78b772629d6b46c8b87cf4f7e0bca276_SeqId_44_] to archive 2023-03-27 03:54:00,735 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:00,738 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/f0342138911442a58f70353e9ff8f0cf to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/f0342138911442a58f70353e9ff8f0cf 2023-03-27 03:54:00,742 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/c4c5ab5e50694399bb74238bd441632c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/c4c5ab5e50694399bb74238bd441632c 2023-03-27 03:54:00,743 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/9bfb49bf600b469485303f963ac55d5f to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/9bfb49bf600b469485303f963ac55d5f 2023-03-27 03:54:00,746 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/d7b466865d6b4ee59ae030d50e592ccd to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/d7b466865d6b4ee59ae030d50e592ccd 2023-03-27 03:54:00,747 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] throttle.PressureAwareThroughputController(145): 78fc96d1b1d88244802e871a9fbedaf3#family1#compaction#23 average throughput is 2.47 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:54:00,748 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/7f0cefa5da584ca499676f67bddafc31 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/7f0cefa5da584ca499676f67bddafc31 2023-03-27 03:54:00,758 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/8ae54e9139644737992ba8e9ad4d0212 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/8ae54e9139644737992ba8e9ad4d0212 2023-03-27 03:54:00,761 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/24ed271a141d42148f4ea5ae25bd0ec9 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/24ed271a141d42148f4ea5ae25bd0ec9 2023-03-27 03:54:00,762 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/220a269f1d56425a821fb7180bfbc712 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/220a269f1d56425a821fb7180bfbc712 2023-03-27 03:54:00,764 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/43dc72ce53f046d3814e5368655de6dd to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/43dc72ce53f046d3814e5368655de6dd 2023-03-27 03:54:00,766 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/78b772629d6b46c8b87cf4f7e0bca276_SeqId_44_ to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/78b772629d6b46c8b87cf4f7e0bca276_SeqId_44_ 2023-03-27 03:54:01,623 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130f78353408a3e4080ae58896f595ce56c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130f78353408a3e4080ae58896f595ce56c 2023-03-27 03:54:01,627 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-27 03:54:01,630 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/.tmp/family1/1245a799e29d495ab2cdbe42bd89cfa7 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/1245a799e29d495ab2cdbe42bd89cfa7 2023-03-27 03:54:01,640 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 78fc96d1b1d88244802e871a9fbedaf3/family1 of 78fc96d1b1d88244802e871a9fbedaf3 into 1245a799e29d495ab2cdbe42bd89cfa7(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:54:01,640 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 78fc96d1b1d88244802e871a9fbedaf3: 2023-03-27 03:54:01,640 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1679889240665; duration=0sec 2023-03-27 03:54:01,640 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:01,669 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/eca5a11168e24706b8171f798662eb61 first=Optional[r00] last=Optional[r8] 2023-03-27 03:54:01,674 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/eca5a11168e24706b8171f798662eb61}] 2023-03-27 03:54:01,677 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/eca5a11168e24706b8171f798662eb61 for inclusion in 78fc96d1b1d88244802e871a9fbedaf3/family1 2023-03-27 03:54:01,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-27 03:54:01,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last= 2023-03-27 03:54:01,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for 78fc96d1b1d88244802e871a9fbedaf3: 2023-03-27 03:54:01,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/eca5a11168e24706b8171f798662eb61 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithMonthlyPolicy__t9igdcsa29drce7nespn3l373qqehlphd50dq375e19rulohh4ajiv70qh9c5pek/family1/eca5a11168e24706b8171f798662eb61 2023-03-27 03:54:01,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithMonthlyPolicy__t9igdcsa29drce7nespn3l373qqehlphd50dq375e19rulohh4ajiv70qh9c5pek/family1/eca5a11168e24706b8171f798662eb61 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/e04bca7e3f844326bad0f614088bfd4a_SeqId_50_ 2023-03-27 03:54:01,685 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithMonthlyPolicy__t9igdcsa29drce7nespn3l373qqehlphd50dq375e19rulohh4ajiv70qh9c5pek/family1/eca5a11168e24706b8171f798662eb61 into 78fc96d1b1d88244802e871a9fbedaf3/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/e04bca7e3f844326bad0f614088bfd4a_SeqId_50_ - updating store file list. 2023-03-27 03:54:01,690 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/e04bca7e3f844326bad0f614088bfd4a_SeqId_50_ into 78fc96d1b1d88244802e871a9fbedaf3/family1 2023-03-27 03:54:01,690 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithMonthlyPolicy__t9igdcsa29drce7nespn3l373qqehlphd50dq375e19rulohh4ajiv70qh9c5pek/family1/eca5a11168e24706b8171f798662eb61 into 78fc96d1b1d88244802e871a9fbedaf3/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/e04bca7e3f844326bad0f614088bfd4a_SeqId_50_) 2023-03-27 03:54:01,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithMonthlyPolicy__t9igdcsa29drce7nespn3l373qqehlphd50dq375e19rulohh4ajiv70qh9c5pek/family1/eca5a11168e24706b8171f798662eb61 2023-03-27 03:54:01,693 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-27 03:54:01,693 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:01,695 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120792552ec7ab948e0823b06728f9f145a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120792552ec7ab948e0823b06728f9f145a 2023-03-27 03:54:01,697 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/e54d0cfe4f754feba36a53a23834c4db, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/9722413088ec482fba0d297ead9903e7, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/ab156cdb3d14444fb159802fa64ebb1c_SeqId_47_] to archive 2023-03-27 03:54:01,697 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:01,700 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/e54d0cfe4f754feba36a53a23834c4db to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/e54d0cfe4f754feba36a53a23834c4db 2023-03-27 03:54:01,701 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112804fc268eb5d64383ad02497bd50260eb to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015112804fc268eb5d64383ad02497bd50260eb 2023-03-27 03:54:01,702 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/9722413088ec482fba0d297ead9903e7 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/9722413088ec482fba0d297ead9903e7 2023-03-27 03:54:01,703 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/ab156cdb3d14444fb159802fa64ebb1c_SeqId_47_ to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/family1/ab156cdb3d14444fb159802fa64ebb1c_SeqId_47_ 2023-03-27 03:54:01,704 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130d0e1250ba7ac402286e5dd20119fb1ec to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130d0e1250ba7ac402286e5dd20119fb1ec 2023-03-27 03:54:01,704 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-27 03:54:01,704 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-27 03:54:01,705 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-27 03:54:01,705 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-27 03:54:01,705 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-03-27 03:54:01,705 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-03-27 03:54:01,705 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:54:01,706 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-27 03:54:01,708 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.6980 sec 2023-03-27 03:54:01,833 INFO [Listener at localhost/36131] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=251 (was 251), OpenFileDescriptor=606 (was 598) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=517 (was 430) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=8923 (was 9102) 2023-03-27 03:54:01,847 INFO [Listener at localhost/36131] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=251, OpenFileDescriptor=606, MaxFileDescriptor=60000, SystemLoadAverage=517, ProcessCount=170, AvailableMemoryMB=8922 2023-03-27 03:54:01,853 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-27 03:54:01,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:01,857 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-27 03:54:01,857 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-03-27 03:54:01,858 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-27 03:54:01,859 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-27 03:54:01,864 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad 2023-03-27 03:54:01,865 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad empty. 2023-03-27 03:54:01,865 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad 2023-03-27 03:54:01,865 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-03-27 03:54:01,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-27 03:54:02,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-27 03:54:02,282 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-27 03:54:02,284 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => eb204466139b909fd8d143a4c79d91ad, NAME => 'testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp 2023-03-27 03:54:02,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-27 03:54:02,693 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:02,693 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing eb204466139b909fd8d143a4c79d91ad, disabling compactions & flushes 2023-03-27 03:54:02,694 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:02,694 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:02,694 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. after waiting 0 ms 2023-03-27 03:54:02,694 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:02,694 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:02,694 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for eb204466139b909fd8d143a4c79d91ad: 2023-03-27 03:54:02,697 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-27 03:54:02,698 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679889242698"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889242698"}]},"ts":"1679889242698"} 2023-03-27 03:54:02,700 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-27 03:54:02,701 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-27 03:54:02,701 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889242701"}]},"ts":"1679889242701"} 2023-03-27 03:54:02,703 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-27 03:54:02,711 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=eb204466139b909fd8d143a4c79d91ad, ASSIGN}] 2023-03-27 03:54:02,713 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=eb204466139b909fd8d143a4c79d91ad, ASSIGN 2023-03-27 03:54:02,714 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=eb204466139b909fd8d143a4c79d91ad, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=false 2023-03-27 03:54:02,865 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=eb204466139b909fd8d143a4c79d91ad, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:02,866 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679889242865"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889242865"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889242865"}]},"ts":"1679889242865"} 2023-03-27 03:54:02,868 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure eb204466139b909fd8d143a4c79d91ad, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:54:02,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-27 03:54:03,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:03,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => eb204466139b909fd8d143a4c79d91ad, NAME => 'testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad.', STARTKEY => '', ENDKEY => ''} 2023-03-27 03:54:03,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy eb204466139b909fd8d143a4c79d91ad 2023-03-27 03:54:03,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:03,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for eb204466139b909fd8d143a4c79d91ad 2023-03-27 03:54:03,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for eb204466139b909fd8d143a4c79d91ad 2023-03-27 03:54:03,026 INFO [StoreOpener-eb204466139b909fd8d143a4c79d91ad-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region eb204466139b909fd8d143a4c79d91ad 2023-03-27 03:54:03,027 DEBUG [StoreOpener-eb204466139b909fd8d143a4c79d91ad-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1 2023-03-27 03:54:03,027 DEBUG [StoreOpener-eb204466139b909fd8d143a4c79d91ad-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1 2023-03-27 03:54:03,027 INFO [StoreOpener-eb204466139b909fd8d143a4c79d91ad-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region eb204466139b909fd8d143a4c79d91ad columnFamilyName family1 2023-03-27 03:54:03,029 INFO [StoreOpener-eb204466139b909fd8d143a4c79d91ad-1] regionserver.HStore(310): Store=eb204466139b909fd8d143a4c79d91ad/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:54:03,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad 2023-03-27 03:54:03,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad 2023-03-27 03:54:03,033 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for eb204466139b909fd8d143a4c79d91ad 2023-03-27 03:54:03,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:54:03,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened eb204466139b909fd8d143a4c79d91ad; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66395565, jitterRate=-0.01062898337841034}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-27 03:54:03,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for eb204466139b909fd8d143a4c79d91ad: 2023-03-27 03:54:03,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad., pid=25, masterSystemTime=1679889243020 2023-03-27 03:54:03,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:03,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:03,041 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=eb204466139b909fd8d143a4c79d91ad, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:03,042 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679889243041"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889243041"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889243041"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889243041"}]},"ts":"1679889243041"} 2023-03-27 03:54:03,045 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-03-27 03:54:03,046 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure eb204466139b909fd8d143a4c79d91ad, server=jenkins-hbase4.apache.org,39727,1679889218560 in 175 msec 2023-03-27 03:54:03,048 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-03-27 03:54:03,048 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=eb204466139b909fd8d143a4c79d91ad, ASSIGN in 336 msec 2023-03-27 03:54:03,053 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-27 03:54:03,054 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889243054"}]},"ts":"1679889243054"} 2023-03-27 03:54:03,055 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-27 03:54:03,057 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-27 03:54:03,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 1.2030 sec 2023-03-27 03:54:03,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-03-27 03:54:03,966 INFO [Listener at localhost/36131] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-03-27 03:54:04,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:04,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,012 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-27 03:54:04,012 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:04,012 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-27 03:54:04,012 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-27 03:54:04,013 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,013 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,014 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,014 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:04,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:04,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:04,015 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,015 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:04,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,016 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:04,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,016 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,020 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,020 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-27 03:54:04,020 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:04,020 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-27 03:54:04,020 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:04,020 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:04,020 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:04,020 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. started... 2023-03-27 03:54:04,021 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing eb204466139b909fd8d143a4c79d91ad 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-27 03:54:04,066 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130f4216b33b2f2470494b0bdbef15dd960 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151130f4216b33b2f2470494b0bdbef15dd960 2023-03-27 03:54:04,074 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/7a94f5ebc5b945a0a3408104db1d9c48 2023-03-27 03:54:04,081 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/7a94f5ebc5b945a0a3408104db1d9c48 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/7a94f5ebc5b945a0a3408104db1d9c48 2023-03-27 03:54:04,088 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/7a94f5ebc5b945a0a3408104db1d9c48, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-27 03:54:04,089 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for eb204466139b909fd8d143a4c79d91ad in 68ms, sequenceid=5, compaction requested=false 2023-03-27 03:54:04,089 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-03-27 03:54:04,090 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for eb204466139b909fd8d143a4c79d91ad: 2023-03-27 03:54:04,090 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:04,090 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:04,090 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:04,090 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,090 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-27 03:54:04,090 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-27 03:54:04,092 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,092 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:04,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:04,092 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,092 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:04,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:04,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:04,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:04,094 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-27 03:54:04,094 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b32e0bf[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:04,094 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-27 03:54:04,094 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,095 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,095 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-27 03:54:04,095 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-27 03:54:04,095 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:04,095 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,095 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:04,103 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,103 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-27 03:54:04,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:04,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:04,103 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:04,103 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-27 03:54:04,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:04,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:04,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:04,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:04,106 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:54:04,106 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-27 03:54:04,106 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:04,106 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:04,106 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,106 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:04,107 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,107 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,108 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:04,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:04,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:04,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:04,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:04,108 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:04,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:04,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,110 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:04,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,112 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,113 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,113 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,113 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:04,113 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,113 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-27 03:54:04,113 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:04,113 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,113 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:04,113 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:04,113 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-27 03:54:04,113 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:04,113 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:04,113 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:04,114 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:04,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:04,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:04,214 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:04,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:04,219 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:04,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:04,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,221 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-27 03:54:04,221 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:04,221 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-27 03:54:04,221 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-27 03:54:04,222 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,222 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,223 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:04,223 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:04,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:04,223 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,224 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:04,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,224 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:04,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,224 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,225 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-27 03:54:04,225 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,225 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-27 03:54:04,225 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:04,225 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-27 03:54:04,226 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:04,226 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:04,226 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:04,226 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. started... 2023-03-27 03:54:04,226 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing eb204466139b909fd8d143a4c79d91ad 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-27 03:54:04,245 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128cdca318a3d5d45ac979877afb652ec81 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151128cdca318a3d5d45ac979877afb652ec81 2023-03-27 03:54:04,254 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/1a520f86d99f4dc0ae913315cf838828 2023-03-27 03:54:04,260 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/1a520f86d99f4dc0ae913315cf838828 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/1a520f86d99f4dc0ae913315cf838828 2023-03-27 03:54:04,265 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/1a520f86d99f4dc0ae913315cf838828, entries=1, sequenceid=9, filesize=5.0 K 2023-03-27 03:54:04,266 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for eb204466139b909fd8d143a4c79d91ad in 40ms, sequenceid=9, compaction requested=false 2023-03-27 03:54:04,266 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for eb204466139b909fd8d143a4c79d91ad: 2023-03-27 03:54:04,266 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:04,266 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:04,266 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:04,266 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,266 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-27 03:54:04,266 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-27 03:54:04,269 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,269 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:04,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:04,269 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,269 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:04,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:04,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:04,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:04,270 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-27 03:54:04,271 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@48332923[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:04,271 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-27 03:54:04,271 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,272 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,272 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-27 03:54:04,272 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-27 03:54:04,272 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,272 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:04,272 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:04,274 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-27 03:54:04,274 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:04,274 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-27 03:54:04,274 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:04,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:04,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:04,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:04,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:04,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:04,277 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:54:04,277 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-27 03:54:04,277 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:04,277 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:04,277 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,277 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:04,279 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,279 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,279 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:04,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:04,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:04,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:04,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:04,280 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:04,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:04,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,281 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:04,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,290 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,290 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,290 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:04,290 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:04,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:04,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:04,290 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-27 03:54:04,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:04,291 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:04,291 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,291 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,291 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,291 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:04,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:04,291 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:04,291 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:04,291 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-27 03:54:04,392 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:04,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:04,397 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:04,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:04,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,399 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-27 03:54:04,399 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:04,400 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-27 03:54:04,400 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-27 03:54:04,400 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,400 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,401 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:04,401 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:04,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:04,402 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,402 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:04,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,402 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:04,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,402 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,403 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-27 03:54:04,403 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,403 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-27 03:54:04,403 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:04,403 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-27 03:54:04,403 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:04,403 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:04,403 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:04,403 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. started... 2023-03-27 03:54:04,403 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing eb204466139b909fd8d143a4c79d91ad 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:04,421 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512013b3cc466740648d1ab068b798f9e6353 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512013b3cc466740648d1ab068b798f9e6353 2023-03-27 03:54:04,433 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/b8de6de49f554a639509156b21509822 2023-03-27 03:54:04,441 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/b8de6de49f554a639509156b21509822 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/b8de6de49f554a639509156b21509822 2023-03-27 03:54:04,446 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/b8de6de49f554a639509156b21509822, entries=1, sequenceid=13, filesize=4.9 K 2023-03-27 03:54:04,447 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for eb204466139b909fd8d143a4c79d91ad in 44ms, sequenceid=13, compaction requested=true 2023-03-27 03:54:04,447 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for eb204466139b909fd8d143a4c79d91ad: 2023-03-27 03:54:04,447 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:04,447 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:04,447 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:04,447 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,447 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-27 03:54:04,447 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-27 03:54:04,449 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,449 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:04,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:04,450 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,450 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:04,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:04,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:04,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:04,451 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-27 03:54:04,451 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@484e458e[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:04,451 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-27 03:54:04,451 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,453 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,453 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-27 03:54:04,453 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-27 03:54:04,453 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:04,453 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,453 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:04,455 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,455 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-27 03:54:04,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:04,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:04,456 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:04,456 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-27 03:54:04,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:04,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:04,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:04,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:04,458 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:54:04,458 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-27 03:54:04,458 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:04,458 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:04,458 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,458 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:04,460 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,460 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:04,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:04,460 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:04,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,460 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:04,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:04,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:04,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:04,464 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:04,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,469 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,469 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,469 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,469 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,469 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,469 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-27 03:54:04,469 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:04,469 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:04,469 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:04,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:04,469 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:04,470 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:04,469 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:04,470 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-27 03:54:04,469 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,469 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:04,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:04,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:04,570 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:04,571 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:04,576 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:04,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:04,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,578 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-27 03:54:04,578 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:04,578 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-27 03:54:04,579 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-27 03:54:04,579 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,579 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,580 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,580 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:04,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:04,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:04,581 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,581 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:04,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,581 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:04,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,582 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,582 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-27 03:54:04,582 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,582 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-27 03:54:04,582 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:04,582 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-27 03:54:04,582 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:04,582 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:04,582 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:04,582 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. started... 2023-03-27 03:54:04,582 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing eb204466139b909fd8d143a4c79d91ad 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:04,597 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205048b756320164765b508a82fcd958c23 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205048b756320164765b508a82fcd958c23 2023-03-27 03:54:04,617 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/d27abde6921f48cd9f5c7c37b047bd08 2023-03-27 03:54:04,623 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/d27abde6921f48cd9f5c7c37b047bd08 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/d27abde6921f48cd9f5c7c37b047bd08 2023-03-27 03:54:04,631 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/d27abde6921f48cd9f5c7c37b047bd08, entries=1, sequenceid=17, filesize=4.9 K 2023-03-27 03:54:04,633 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for eb204466139b909fd8d143a4c79d91ad in 51ms, sequenceid=17, compaction requested=true 2023-03-27 03:54:04,633 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for eb204466139b909fd8d143a4c79d91ad: 2023-03-27 03:54:04,633 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:04,633 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:04,633 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:04,633 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,633 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-27 03:54:04,633 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-27 03:54:04,635 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:04,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:04,635 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:04,635 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,636 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:04,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:04,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:04,637 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-27 03:54:04,637 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25aec69d[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:04,637 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-27 03:54:04,637 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,644 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,644 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-27 03:54:04,644 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-27 03:54:04,644 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:04,644 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,644 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:04,646 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,646 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-27 03:54:04,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:04,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:04,646 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:04,646 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-27 03:54:04,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:04,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:04,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:04,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:04,648 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:54:04,649 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-27 03:54:04,649 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:04,649 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:04,649 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,649 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:04,650 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,650 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,650 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:04,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:04,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:04,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:04,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:04,651 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:04,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:04,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,653 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:04,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,656 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:04,656 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:04,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:04,656 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:04,656 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,656 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,656 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,656 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,656 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,656 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:04,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:04,659 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:04,659 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-27 03:54:04,659 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:04,659 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:04,660 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:04,760 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:04,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:04,765 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:04,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:04,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,767 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-27 03:54:04,767 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:04,767 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-27 03:54:04,767 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-27 03:54:04,767 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,767 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,770 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,770 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:04,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:04,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:04,770 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:04,770 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:04,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,770 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:04,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,771 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,771 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-27 03:54:04,771 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:04,771 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-27 03:54:04,771 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:04,771 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-27 03:54:04,771 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:04,771 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:04,771 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:04,771 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. started... 2023-03-27 03:54:04,771 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing eb204466139b909fd8d143a4c79d91ad 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:05,185 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228a0796ebfcb5e4f8abd810c347f46addf to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228a0796ebfcb5e4f8abd810c347f46addf 2023-03-27 03:54:05,590 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/60a649de349f4fbcb8b6d5241443a594 2023-03-27 03:54:05,596 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/60a649de349f4fbcb8b6d5241443a594 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/60a649de349f4fbcb8b6d5241443a594 2023-03-27 03:54:05,600 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/60a649de349f4fbcb8b6d5241443a594, entries=1, sequenceid=21, filesize=4.9 K 2023-03-27 03:54:05,601 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for eb204466139b909fd8d143a4c79d91ad in 830ms, sequenceid=21, compaction requested=true 2023-03-27 03:54:05,601 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for eb204466139b909fd8d143a4c79d91ad: 2023-03-27 03:54:05,601 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:05,601 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:05,602 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:05,602 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:05,602 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-27 03:54:05,602 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-27 03:54:05,603 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:05,603 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:05,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:05,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:05,604 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,604 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:05,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:05,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:05,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:05,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:05,605 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-27 03:54:05,605 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6955ed8f[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:05,605 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-27 03:54:05,605 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,606 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,606 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-27 03:54:05,606 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-27 03:54:05,606 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:05,606 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:05,606 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:05,607 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:05,607 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-27 03:54:05,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:05,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:05,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:05,607 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:05,607 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-27 03:54:05,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:05,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:05,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:05,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:05,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:05,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:05,610 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:54:05,610 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-27 03:54:05,610 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:05,610 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:05,610 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,610 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:05,612 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,612 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,612 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:05,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:05,612 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:05,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:05,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:05,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:05,613 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:05,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:05,615 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:05,615 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,615 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,615 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:05,615 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,615 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:05,615 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:05,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-27 03:54:05,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:05,615 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,615 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:05,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:05,616 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-27 03:54:05,616 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:05,616 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:05,616 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:05,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:05,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:05,716 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:05,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:05,722 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:05,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:05,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,723 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-27 03:54:05,723 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:05,723 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-27 03:54:05,723 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-27 03:54:05,724 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,724 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,725 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:05,725 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:05,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:05,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:05,725 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:05,725 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:05,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,726 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:05,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,726 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,726 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-27 03:54:05,726 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:05,726 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-27 03:54:05,727 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:05,727 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-27 03:54:05,727 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:05,727 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:05,727 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:05,727 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. started... 2023-03-27 03:54:05,727 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing eb204466139b909fd8d143a4c79d91ad 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:06,140 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512319e3a51de08bc4584b28f2fefda37d779 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512319e3a51de08bc4584b28f2fefda37d779 2023-03-27 03:54:06,546 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/a78fad4ce4544afd9edb3dabe7bb6c16 2023-03-27 03:54:06,551 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/a78fad4ce4544afd9edb3dabe7bb6c16 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/a78fad4ce4544afd9edb3dabe7bb6c16 2023-03-27 03:54:06,555 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/a78fad4ce4544afd9edb3dabe7bb6c16, entries=1, sequenceid=25, filesize=4.9 K 2023-03-27 03:54:06,556 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for eb204466139b909fd8d143a4c79d91ad in 829ms, sequenceid=25, compaction requested=true 2023-03-27 03:54:06,556 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for eb204466139b909fd8d143a4c79d91ad: 2023-03-27 03:54:06,556 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:06,557 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:06,557 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:06,557 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:06,557 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-27 03:54:06,557 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-27 03:54:06,558 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:06,558 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:06,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:06,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:06,558 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,559 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:06,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:06,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:06,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:06,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:06,560 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-27 03:54:06,560 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38febbb7[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:06,560 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-27 03:54:06,560 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,562 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,562 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-27 03:54:06,562 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-27 03:54:06,562 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:06,562 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:06,563 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:06,566 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-27 03:54:06,566 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:06,566 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:06,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:06,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:06,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:06,566 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-27 03:54:06,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:06,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:06,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:06,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:06,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:06,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:06,569 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:54:06,569 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-27 03:54:06,569 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:06,569 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:06,569 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,569 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:06,570 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,570 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:06,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:06,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,570 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:06,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:06,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:06,571 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:06,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:06,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:06,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,572 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:06,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:06,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:06,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:06,574 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:06,574 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,574 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:06,574 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:06,574 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:06,574 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:06,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:06,574 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,574 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-27 03:54:06,574 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,574 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:06,574 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,574 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:06,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:06,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:06,574 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:06,575 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:06,675 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:06,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:06,679 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:06,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:06,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,681 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-27 03:54:06,681 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:06,681 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-27 03:54:06,681 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-27 03:54:06,682 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,682 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,683 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:06,683 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:06,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:06,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:06,683 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:06,683 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:06,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,683 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:06,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,684 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,684 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-27 03:54:06,684 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:06,684 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-27 03:54:06,684 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:06,684 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-27 03:54:06,684 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:06,684 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:06,684 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:06,684 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. started... 2023-03-27 03:54:06,684 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing eb204466139b909fd8d143a4c79d91ad 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:07,099 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101eea29cc0da7d48d7b0f561be1b86cae5 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101eea29cc0da7d48d7b0f561be1b86cae5 2023-03-27 03:54:07,505 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/1430575755534c68a4959243c8e65b7c 2023-03-27 03:54:07,510 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/1430575755534c68a4959243c8e65b7c as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/1430575755534c68a4959243c8e65b7c 2023-03-27 03:54:07,514 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/1430575755534c68a4959243c8e65b7c, entries=1, sequenceid=29, filesize=4.9 K 2023-03-27 03:54:07,515 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for eb204466139b909fd8d143a4c79d91ad in 831ms, sequenceid=29, compaction requested=true 2023-03-27 03:54:07,515 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for eb204466139b909fd8d143a4c79d91ad: 2023-03-27 03:54:07,515 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:07,515 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:07,515 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:07,515 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:07,516 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-27 03:54:07,516 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-27 03:54:07,517 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:07,517 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:07,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:07,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:07,517 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,517 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:07,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:07,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:07,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:07,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:07,519 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-27 03:54:07,519 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@226eb13[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:07,519 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-27 03:54:07,519 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,520 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,520 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-27 03:54:07,520 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-27 03:54:07,520 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:07,520 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:07,520 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:07,521 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:07,521 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-27 03:54:07,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:07,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:07,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:07,521 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:07,521 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-27 03:54:07,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:07,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:07,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:07,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:07,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:07,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:07,523 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:54:07,523 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-27 03:54:07,523 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:07,523 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:07,523 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,523 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:07,525 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,525 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:07,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:07,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,525 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:07,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:07,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:07,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:07,526 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:07,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:07,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:07,527 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:07,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:07,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:07,529 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:07,529 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:07,529 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:07,529 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-27 03:54:07,529 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,529 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-27 03:54:07,529 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:07,529 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:07,529 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:07,529 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:07,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:07,529 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,529 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,530 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,530 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:07,530 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:07,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:07,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:07,630 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:07,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:07,635 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:07,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:07,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,637 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-27 03:54:07,637 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:07,637 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-27 03:54:07,637 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-27 03:54:07,637 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,637 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,638 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:07,638 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:07,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:07,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:07,638 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:07,638 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:07,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,639 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:07,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,639 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,639 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-27 03:54:07,639 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:07,639 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-27 03:54:07,639 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:07,640 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-27 03:54:07,640 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:07,640 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:07,640 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:07,640 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. started... 2023-03-27 03:54:07,640 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing eb204466139b909fd8d143a4c79d91ad 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:08,053 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601039ce5fdc5c82849e18beb26dfa49009fc to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601039ce5fdc5c82849e18beb26dfa49009fc 2023-03-27 03:54:08,458 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/256508fa14384b66bbb654851d533eda 2023-03-27 03:54:08,464 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/256508fa14384b66bbb654851d533eda as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/256508fa14384b66bbb654851d533eda 2023-03-27 03:54:08,468 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/256508fa14384b66bbb654851d533eda, entries=1, sequenceid=33, filesize=4.9 K 2023-03-27 03:54:08,469 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for eb204466139b909fd8d143a4c79d91ad in 829ms, sequenceid=33, compaction requested=true 2023-03-27 03:54:08,470 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for eb204466139b909fd8d143a4c79d91ad: 2023-03-27 03:54:08,470 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:08,470 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:08,470 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:08,470 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:08,470 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-27 03:54:08,470 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-27 03:54:08,471 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:08,471 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:08,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:08,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:08,472 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,472 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:08,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:08,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:08,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:08,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:08,473 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-27 03:54:08,473 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@51ef4fe4[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:08,473 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-27 03:54:08,473 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,475 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,476 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-27 03:54:08,476 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-27 03:54:08,476 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:08,476 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:08,476 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:08,477 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-27 03:54:08,477 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:08,477 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:08,477 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-27 03:54:08,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:08,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:08,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:08,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:08,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:08,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:08,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:08,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:08,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:08,479 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:54:08,479 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-27 03:54:08,479 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:08,479 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:08,479 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,479 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:08,481 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,481 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,481 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:08,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:08,481 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:08,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,482 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:08,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:08,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:08,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:08,484 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:08,484 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,484 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,484 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:08,484 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,484 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,484 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,484 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-27 03:54:08,484 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:08,484 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-27 03:54:08,484 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:08,484 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:08,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:08,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:08,484 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:08,485 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:08,485 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:08,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:08,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:08,549 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-27 03:54:08,585 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:08,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:08,590 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:08,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:08,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,591 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-27 03:54:08,592 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:08,592 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-27 03:54:08,592 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-27 03:54:08,592 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,592 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,598 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:08,598 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:08,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:08,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:08,598 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:08,598 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:08,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,598 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:08,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,599 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,599 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-27 03:54:08,599 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:08,599 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-27 03:54:08,599 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:08,599 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-27 03:54:08,599 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:08,599 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:08,599 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:08,599 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. started... 2023-03-27 03:54:08,599 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing eb204466139b909fd8d143a4c79d91ad 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:09,012 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509071193ab41b9c1410bb3dc0e4f0e04a6ed to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201509071193ab41b9c1410bb3dc0e4f0e04a6ed 2023-03-27 03:54:09,418 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/0bcd7265ed884be895612e015e20ca9b 2023-03-27 03:54:09,423 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/0bcd7265ed884be895612e015e20ca9b as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/0bcd7265ed884be895612e015e20ca9b 2023-03-27 03:54:09,428 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/0bcd7265ed884be895612e015e20ca9b, entries=1, sequenceid=37, filesize=4.9 K 2023-03-27 03:54:09,429 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for eb204466139b909fd8d143a4c79d91ad in 830ms, sequenceid=37, compaction requested=true 2023-03-27 03:54:09,429 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for eb204466139b909fd8d143a4c79d91ad: 2023-03-27 03:54:09,429 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:09,429 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:09,429 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:09,429 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:09,429 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-27 03:54:09,429 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-27 03:54:09,431 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:09,431 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:09,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:09,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:09,431 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,431 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:09,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:09,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:09,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:09,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:09,433 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-27 03:54:09,433 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1790f9c3[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:09,433 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-27 03:54:09,433 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,437 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,437 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-27 03:54:09,437 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-27 03:54:09,437 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:09,437 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:09,437 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:09,440 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:09,440 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-27 03:54:09,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:09,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:09,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:09,440 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:09,440 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-27 03:54:09,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:09,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:09,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:09,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:09,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:09,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:09,442 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:54:09,442 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-27 03:54:09,442 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:09,442 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:09,442 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,443 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:09,444 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,444 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:09,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:09,444 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:09,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,444 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:09,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:09,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:09,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:09,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:09,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,445 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:09,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:09,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:09,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:09,447 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:09,447 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:09,447 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,447 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:09,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-27 03:54:09,448 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:09,447 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:09,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:09,447 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:09,448 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:09,448 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:09,448 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:09,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:09,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:09,448 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,448 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,448 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,548 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:09,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:09,553 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:09,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:09,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,555 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-03-27 03:54:09,555 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:09,555 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-27 03:54:09,555 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-27 03:54:09,556 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,556 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,557 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:09,557 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:09,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:09,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:09,557 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:09,557 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:09,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,557 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:09,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,558 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,558 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-03-27 03:54:09,558 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:09,558 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-27 03:54:09,558 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:09,558 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-27 03:54:09,558 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:09,558 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:09,558 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:09,558 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. started... 2023-03-27 03:54:09,559 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing eb204466139b909fd8d143a4c79d91ad 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:09,972 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120cc8018ae86be4b6993244d545f1c5e3f to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151120cc8018ae86be4b6993244d545f1c5e3f 2023-03-27 03:54:10,378 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/8d888ae77aea486c9d322ccb7c36c2f3 2023-03-27 03:54:10,383 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/8d888ae77aea486c9d322ccb7c36c2f3 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/8d888ae77aea486c9d322ccb7c36c2f3 2023-03-27 03:54:10,387 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/8d888ae77aea486c9d322ccb7c36c2f3, entries=1, sequenceid=41, filesize=4.9 K 2023-03-27 03:54:10,388 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for eb204466139b909fd8d143a4c79d91ad in 829ms, sequenceid=41, compaction requested=true 2023-03-27 03:54:10,388 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for eb204466139b909fd8d143a4c79d91ad: 2023-03-27 03:54:10,389 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:10,389 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:10,389 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:10,389 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:10,389 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-03-27 03:54:10,389 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-03-27 03:54:10,391 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:10,391 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:10,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:10,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:10,392 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,392 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:10,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:10,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:10,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:10,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:10,393 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-03-27 03:54:10,393 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a7ecb85[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:10,393 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-27 03:54:10,393 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,394 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,395 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-27 03:54:10,395 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-03-27 03:54:10,395 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:10,395 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:10,395 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:10,396 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-27 03:54:10,396 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:10,396 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:10,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:10,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:10,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:10,396 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-03-27 03:54:10,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:10,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:10,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:10,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:10,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:10,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:10,398 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:54:10,398 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-03-27 03:54:10,398 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:10,398 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:10,398 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,398 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:10,400 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,400 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:10,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:10,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,400 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:10,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:10,400 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:10,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:10,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:10,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:10,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,401 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:10,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:10,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:10,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:10,404 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:10,404 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:10,404 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:10,404 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:10,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:10,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:10,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-03-27 03:54:10,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:10,404 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,404 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,404 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:10,404 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,404 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,404 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:10,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:10,405 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:10,405 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:10,505 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:10,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:10,507 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38271] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-03-27 03:54:10,508 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-27 03:54:10,509 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-27 03:54:10,510 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:54:10,510 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679889850510 2023-03-27 03:54:10,511 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-27 03:54:10,512 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-27 03:54:10,512 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-27 03:54:10,513 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] zookeeper.ReadOnlyZKClient(139): Connect 0x4ff36841 to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:54:10,517 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@29002264, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:54:10,517 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-27 03:54:11,384 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103b6b947e92a4e48c193db11f395a52d57 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103b6b947e92a4e48c193db11f395a52d57 2023-03-27 03:54:11,386 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-27 03:54:11,413 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-27 03:54:11,414 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42970, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-27 03:54:11,453 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-27 03:54:11,455 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34510, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-27 03:54:11,468 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/c00af26c944b48538bf93f166115eb21 first=Optional[r3] last=Optional[r6] 2023-03-27 03:54:11,468 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/c00af26c944b48538bf93f166115eb21}] 2023-03-27 03:54:11,475 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/c00af26c944b48538bf93f166115eb21 for inclusion in eb204466139b909fd8d143a4c79d91ad/family1 2023-03-27 03:54:11,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-27 03:54:11,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last= 2023-03-27 03:54:11,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for eb204466139b909fd8d143a4c79d91ad: 2023-03-27 03:54:11,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/c00af26c944b48538bf93f166115eb21 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithWeeklyPolicy__j5i1j1ffvq6src1aa4t9omt21dv9qu271usph727q5sr785c074n7q7phnuqajt9/family1/c00af26c944b48538bf93f166115eb21 2023-03-27 03:54:11,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithWeeklyPolicy__j5i1j1ffvq6src1aa4t9omt21dv9qu271usph727q5sr785c074n7q7phnuqajt9/family1/c00af26c944b48538bf93f166115eb21 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/ba472d18ed764c6fbbfebbe2d1f2d3b8_SeqId_44_ 2023-03-27 03:54:11,484 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithWeeklyPolicy__j5i1j1ffvq6src1aa4t9omt21dv9qu271usph727q5sr785c074n7q7phnuqajt9/family1/c00af26c944b48538bf93f166115eb21 into eb204466139b909fd8d143a4c79d91ad/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/ba472d18ed764c6fbbfebbe2d1f2d3b8_SeqId_44_ - updating store file list. 2023-03-27 03:54:11,496 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/ba472d18ed764c6fbbfebbe2d1f2d3b8_SeqId_44_ into eb204466139b909fd8d143a4c79d91ad/family1 2023-03-27 03:54:11,496 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithWeeklyPolicy__j5i1j1ffvq6src1aa4t9omt21dv9qu271usph727q5sr785c074n7q7phnuqajt9/family1/c00af26c944b48538bf93f166115eb21 into eb204466139b909fd8d143a4c79d91ad/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/ba472d18ed764c6fbbfebbe2d1f2d3b8_SeqId_44_) 2023-03-27 03:54:11,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithWeeklyPolicy__j5i1j1ffvq6src1aa4t9omt21dv9qu271usph727q5sr785c074n7q7phnuqajt9/family1/c00af26c944b48538bf93f166115eb21 2023-03-27 03:54:11,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-27 03:54:11,508 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-27 03:54:11,508 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(1912): eb204466139b909fd8d143a4c79d91ad/family1 is initiating minor compaction 2023-03-27 03:54:11,508 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@ae2ed52; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:11,508 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of eb204466139b909fd8d143a4c79d91ad/family1 in testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:11,508 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/1a520f86d99f4dc0ae913315cf838828, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/b8de6de49f554a639509156b21509822, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/d27abde6921f48cd9f5c7c37b047bd08, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/60a649de349f4fbcb8b6d5241443a594, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/a78fad4ce4544afd9edb3dabe7bb6c16, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/1430575755534c68a4959243c8e65b7c, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/256508fa14384b66bbb654851d533eda, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/0bcd7265ed884be895612e015e20ca9b, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/8d888ae77aea486c9d322ccb7c36c2f3, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/ba472d18ed764c6fbbfebbe2d1f2d3b8_SeqId_44_] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp, totalSize=50.0 K 2023-03-27 03:54:11,509 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 1a520f86d99f4dc0ae913315cf838828, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-27 03:54:11,509 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting b8de6de49f554a639509156b21509822, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-27 03:54:11,509 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting d27abde6921f48cd9f5c7c37b047bd08, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-27 03:54:11,509 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 60a649de349f4fbcb8b6d5241443a594, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-27 03:54:11,509 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting a78fad4ce4544afd9edb3dabe7bb6c16, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-27 03:54:11,509 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 1430575755534c68a4959243c8e65b7c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-27 03:54:11,509 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 256508fa14384b66bbb654851d533eda, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-27 03:54:11,509 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 0bcd7265ed884be895612e015e20ca9b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-27 03:54:11,509 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 8d888ae77aea486c9d322ccb7c36c2f3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-27 03:54:11,509 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting ba472d18ed764c6fbbfebbe2d1f2d3b8_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-27 03:54:11,512 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-27 03:54:11,513 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:11,516 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228a0796ebfcb5e4f8abd810c347f46addf to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228a0796ebfcb5e4f8abd810c347f46addf 2023-03-27 03:54:11,518 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512319e3a51de08bc4584b28f2fefda37d779 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512319e3a51de08bc4584b28f2fefda37d779 2023-03-27 03:54:11,521 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101eea29cc0da7d48d7b0f561be1b86cae5 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101eea29cc0da7d48d7b0f561be1b86cae5 2023-03-27 03:54:11,523 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601039ce5fdc5c82849e18beb26dfa49009fc to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601039ce5fdc5c82849e18beb26dfa49009fc 2023-03-27 03:54:11,523 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-27 03:54:11,523 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-27 03:54:11,524 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-27 03:54:11,524 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-27 03:54:11,524 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-27 03:54:11,524 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-27 03:54:11,524 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-27 03:54:11,524 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-27 03:54:11,524 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-27 03:54:11,525 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-27 03:54:11,593 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] throttle.PressureAwareThroughputController(145): eb204466139b909fd8d143a4c79d91ad#family1#compaction#34 average throughput is 0.45 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:54:11,627 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205e700f3d367774ec390b18bad136117a2 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205e700f3d367774ec390b18bad136117a2 2023-03-27 03:54:11,629 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-27 03:54:11,650 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/32be9b439d75468f9012d99bd78db52a as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/32be9b439d75468f9012d99bd78db52a 2023-03-27 03:54:11,657 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in eb204466139b909fd8d143a4c79d91ad/family1 of eb204466139b909fd8d143a4c79d91ad into 32be9b439d75468f9012d99bd78db52a(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:54:11,657 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for eb204466139b909fd8d143a4c79d91ad: 2023-03-27 03:54:11,657 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1679889251508; duration=0sec 2023-03-27 03:54:11,658 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:11,677 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b36ba710d8914dafba466680b4d2921a first=Optional[r1] last=Optional[r2] 2023-03-27 03:54:11,677 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b36ba710d8914dafba466680b4d2921a}] 2023-03-27 03:54:11,682 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b36ba710d8914dafba466680b4d2921a for inclusion in eb204466139b909fd8d143a4c79d91ad/family1 2023-03-27 03:54:11,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-27 03:54:11,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last= 2023-03-27 03:54:11,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for eb204466139b909fd8d143a4c79d91ad: 2023-03-27 03:54:11,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/b36ba710d8914dafba466680b4d2921a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithWeeklyPolicy__dmghdh8256im738sp0fl04sr332a46le80mvl6u0vgadm0moeesrush08q5tiscn/family1/b36ba710d8914dafba466680b4d2921a 2023-03-27 03:54:11,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithWeeklyPolicy__dmghdh8256im738sp0fl04sr332a46le80mvl6u0vgadm0moeesrush08q5tiscn/family1/b36ba710d8914dafba466680b4d2921a as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/5abcc73cec944ccf968184aa9a5a40a1_SeqId_47_ 2023-03-27 03:54:11,692 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithWeeklyPolicy__dmghdh8256im738sp0fl04sr332a46le80mvl6u0vgadm0moeesrush08q5tiscn/family1/b36ba710d8914dafba466680b4d2921a into eb204466139b909fd8d143a4c79d91ad/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/5abcc73cec944ccf968184aa9a5a40a1_SeqId_47_ - updating store file list. 2023-03-27 03:54:11,707 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/1a520f86d99f4dc0ae913315cf838828, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/b8de6de49f554a639509156b21509822, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/d27abde6921f48cd9f5c7c37b047bd08, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/60a649de349f4fbcb8b6d5241443a594, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/a78fad4ce4544afd9edb3dabe7bb6c16, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/1430575755534c68a4959243c8e65b7c, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/256508fa14384b66bbb654851d533eda, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/0bcd7265ed884be895612e015e20ca9b, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/8d888ae77aea486c9d322ccb7c36c2f3, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/ba472d18ed764c6fbbfebbe2d1f2d3b8_SeqId_44_] to archive 2023-03-27 03:54:11,709 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:11,709 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/5abcc73cec944ccf968184aa9a5a40a1_SeqId_47_ into eb204466139b909fd8d143a4c79d91ad/family1 2023-03-27 03:54:11,709 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithWeeklyPolicy__dmghdh8256im738sp0fl04sr332a46le80mvl6u0vgadm0moeesrush08q5tiscn/family1/b36ba710d8914dafba466680b4d2921a into eb204466139b909fd8d143a4c79d91ad/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/5abcc73cec944ccf968184aa9a5a40a1_SeqId_47_) 2023-03-27 03:54:11,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithWeeklyPolicy__dmghdh8256im738sp0fl04sr332a46le80mvl6u0vgadm0moeesrush08q5tiscn/family1/b36ba710d8914dafba466680b4d2921a 2023-03-27 03:54:11,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-27 03:54:11,715 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/1a520f86d99f4dc0ae913315cf838828 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/1a520f86d99f4dc0ae913315cf838828 2023-03-27 03:54:11,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 181014 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-27 03:54:11,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(1912): eb204466139b909fd8d143a4c79d91ad/family1 is initiating minor compaction (all files) 2023-03-27 03:54:11,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2ff62394; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:11,716 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of eb204466139b909fd8d143a4c79d91ad/family1 in testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:54:11,716 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/7a94f5ebc5b945a0a3408104db1d9c48, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/32be9b439d75468f9012d99bd78db52a, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/5abcc73cec944ccf968184aa9a5a40a1_SeqId_47_] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp, totalSize=176.8 K 2023-03-27 03:54:11,717 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 7a94f5ebc5b945a0a3408104db1d9c48, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878842162 2023-03-27 03:54:11,718 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/b8de6de49f554a639509156b21509822 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/b8de6de49f554a639509156b21509822 2023-03-27 03:54:11,720 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/d27abde6921f48cd9f5c7c37b047bd08 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/d27abde6921f48cd9f5c7c37b047bd08 2023-03-27 03:54:11,720 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 32be9b439d75468f9012d99bd78db52a, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621242162 2023-03-27 03:54:11,720 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 5abcc73cec944ccf968184aa9a5a40a1_SeqId_47_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965242162 2023-03-27 03:54:11,721 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-27 03:54:11,721 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/60a649de349f4fbcb8b6d5241443a594 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/60a649de349f4fbcb8b6d5241443a594 2023-03-27 03:54:11,722 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:11,724 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512013b3cc466740648d1ab068b798f9e6353 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512013b3cc466740648d1ab068b798f9e6353 2023-03-27 03:54:11,724 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/a78fad4ce4544afd9edb3dabe7bb6c16 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/a78fad4ce4544afd9edb3dabe7bb6c16 2023-03-27 03:54:11,726 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205048b756320164765b508a82fcd958c23 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205048b756320164765b508a82fcd958c23 2023-03-27 03:54:11,726 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-03-27 03:54:11,726 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-27 03:54:11,727 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-03-27 03:54:11,727 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-03-27 03:54:11,727 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:54:11,728 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-27 03:54:11,728 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/1430575755534c68a4959243c8e65b7c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/1430575755534c68a4959243c8e65b7c 2023-03-27 03:54:11,731 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.2210 sec 2023-03-27 03:54:11,732 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/256508fa14384b66bbb654851d533eda to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/256508fa14384b66bbb654851d533eda 2023-03-27 03:54:11,734 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/0bcd7265ed884be895612e015e20ca9b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/0bcd7265ed884be895612e015e20ca9b 2023-03-27 03:54:11,735 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/8d888ae77aea486c9d322ccb7c36c2f3 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/8d888ae77aea486c9d322ccb7c36c2f3 2023-03-27 03:54:11,740 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/ba472d18ed764c6fbbfebbe2d1f2d3b8_SeqId_44_ to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/ba472d18ed764c6fbbfebbe2d1f2d3b8_SeqId_44_ 2023-03-27 03:54:11,753 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] throttle.PressureAwareThroughputController(145): eb204466139b909fd8d143a4c79d91ad#family1#compaction#35 average throughput is 11.90 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:54:11,875 INFO [Listener at localhost/36131] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=256 (was 251) - Thread LEAK? -, OpenFileDescriptor=599 (was 606), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=445 (was 517), ProcessCount=170 (was 170), AvailableMemoryMB=8910 (was 8922) 2023-03-27 03:54:11,889 INFO [Listener at localhost/36131] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=256, OpenFileDescriptor=599, MaxFileDescriptor=60000, SystemLoadAverage=445, ProcessCount=170, AvailableMemoryMB=8909 2023-03-27 03:54:11,892 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-27 03:54:11,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:11,895 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-03-27 03:54:11,895 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-03-27 03:54:11,896 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-27 03:54:11,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-27 03:54:11,901 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testScannerOnBulkLoadRefHFiles/c0982fa95f3b2e90e5d67d21d9f32707 2023-03-27 03:54:11,901 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testScannerOnBulkLoadRefHFiles/1002361bf1e6585024f86241020836f1 2023-03-27 03:54:11,901 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6 2023-03-27 03:54:11,902 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testScannerOnBulkLoadRefHFiles/c0982fa95f3b2e90e5d67d21d9f32707 empty. 2023-03-27 03:54:11,902 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testScannerOnBulkLoadRefHFiles/1002361bf1e6585024f86241020836f1 empty. 2023-03-27 03:54:11,902 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6 empty. 2023-03-27 03:54:11,902 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testScannerOnBulkLoadRefHFiles/c0982fa95f3b2e90e5d67d21d9f32707 2023-03-27 03:54:11,902 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6 2023-03-27 03:54:11,902 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testScannerOnBulkLoadRefHFiles/1002361bf1e6585024f86241020836f1 2023-03-27 03:54:11,902 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-03-27 03:54:11,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-27 03:54:12,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-27 03:54:12,320 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-03-27 03:54:12,324 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7675): creating {ENCODED => 1002361bf1e6585024f86241020836f1, NAME => 'testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp 2023-03-27 03:54:12,324 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7675): creating {ENCODED => c0982fa95f3b2e90e5d67d21d9f32707, NAME => 'testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp 2023-03-27 03:54:12,324 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7675): creating {ENCODED => f2819f01ee4f081befffef519bfab5a6, NAME => 'testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp 2023-03-27 03:54:12,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-27 03:54:12,600 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/.tmp/family1/9c31dc982efd4616b54051bbe32d7d6c as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/9c31dc982efd4616b54051bbe32d7d6c 2023-03-27 03:54:12,606 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in eb204466139b909fd8d143a4c79d91ad/family1 of eb204466139b909fd8d143a4c79d91ad into 9c31dc982efd4616b54051bbe32d7d6c(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:54:12,606 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for eb204466139b909fd8d143a4c79d91ad: 2023-03-27 03:54:12,606 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad., storeName=family1, fileCount=3, fileSize=176.8 K (165.1 K, 6.6 K, 5.2 K), priority=2, time=1679889251716; duration=0sec 2023-03-27 03:54:12,606 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:12,697 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/7a94f5ebc5b945a0a3408104db1d9c48, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/32be9b439d75468f9012d99bd78db52a, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/5abcc73cec944ccf968184aa9a5a40a1_SeqId_47_] to archive 2023-03-27 03:54:12,698 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:12,701 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/7a94f5ebc5b945a0a3408104db1d9c48 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/7a94f5ebc5b945a0a3408104db1d9c48 2023-03-27 03:54:12,703 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/32be9b439d75468f9012d99bd78db52a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/32be9b439d75468f9012d99bd78db52a 2023-03-27 03:54:12,705 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/5abcc73cec944ccf968184aa9a5a40a1_SeqId_47_ to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/family1/5abcc73cec944ccf968184aa9a5a40a1_SeqId_47_ 2023-03-27 03:54:12,739 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:12,739 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1604): Closing c0982fa95f3b2e90e5d67d21d9f32707, disabling compactions & flushes 2023-03-27 03:54:12,739 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707. 2023-03-27 03:54:12,739 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707. 2023-03-27 03:54:12,739 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707. after waiting 0 ms 2023-03-27 03:54:12,739 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707. 2023-03-27 03:54:12,739 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707. 2023-03-27 03:54:12,739 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1558): Region close journal for c0982fa95f3b2e90e5d67d21d9f32707: 2023-03-27 03:54:12,740 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:12,740 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1604): Closing 1002361bf1e6585024f86241020836f1, disabling compactions & flushes 2023-03-27 03:54:12,740 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1. 2023-03-27 03:54:12,740 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1. 2023-03-27 03:54:12,740 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1. after waiting 0 ms 2023-03-27 03:54:12,740 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1. 2023-03-27 03:54:12,741 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1. 2023-03-27 03:54:12,741 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1558): Region close journal for 1002361bf1e6585024f86241020836f1: 2023-03-27 03:54:12,741 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:12,741 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1604): Closing f2819f01ee4f081befffef519bfab5a6, disabling compactions & flushes 2023-03-27 03:54:12,741 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6. 2023-03-27 03:54:12,741 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6. 2023-03-27 03:54:12,741 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6. after waiting 0 ms 2023-03-27 03:54:12,741 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6. 2023-03-27 03:54:12,741 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6. 2023-03-27 03:54:12,741 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1558): Region close journal for f2819f01ee4f081befffef519bfab5a6: 2023-03-27 03:54:12,745 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-03-27 03:54:12,746 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679889252746"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889252746"}]},"ts":"1679889252746"} 2023-03-27 03:54:12,746 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1679889252746"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889252746"}]},"ts":"1679889252746"} 2023-03-27 03:54:12,746 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679889252746"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889252746"}]},"ts":"1679889252746"} 2023-03-27 03:54:12,749 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-27 03:54:12,750 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-27 03:54:12,750 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889252750"}]},"ts":"1679889252750"} 2023-03-27 03:54:12,752 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-03-27 03:54:12,755 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=c0982fa95f3b2e90e5d67d21d9f32707, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=1002361bf1e6585024f86241020836f1, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=f2819f01ee4f081befffef519bfab5a6, ASSIGN}] 2023-03-27 03:54:12,757 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=c0982fa95f3b2e90e5d67d21d9f32707, ASSIGN 2023-03-27 03:54:12,758 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=f2819f01ee4f081befffef519bfab5a6, ASSIGN 2023-03-27 03:54:12,758 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=1002361bf1e6585024f86241020836f1, ASSIGN 2023-03-27 03:54:12,759 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=c0982fa95f3b2e90e5d67d21d9f32707, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=false 2023-03-27 03:54:12,759 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=f2819f01ee4f081befffef519bfab5a6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=false 2023-03-27 03:54:12,759 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=1002361bf1e6585024f86241020836f1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=false 2023-03-27 03:54:12,911 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=c0982fa95f3b2e90e5d67d21d9f32707, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:12,911 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=f2819f01ee4f081befffef519bfab5a6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:12,911 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=1002361bf1e6585024f86241020836f1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:12,911 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679889252911"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889252911"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889252911"}]},"ts":"1679889252911"} 2023-03-27 03:54:12,911 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679889252911"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889252911"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889252911"}]},"ts":"1679889252911"} 2023-03-27 03:54:12,911 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1679889252911"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889252911"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889252911"}]},"ts":"1679889252911"} 2023-03-27 03:54:12,913 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE; OpenRegionProcedure c0982fa95f3b2e90e5d67d21d9f32707, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:54:12,914 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=30, state=RUNNABLE; OpenRegionProcedure f2819f01ee4f081befffef519bfab5a6, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:54:12,915 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=29, state=RUNNABLE; OpenRegionProcedure 1002361bf1e6585024f86241020836f1, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:54:13,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-27 03:54:13,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707. 2023-03-27 03:54:13,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c0982fa95f3b2e90e5d67d21d9f32707, NAME => 'testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707.', STARTKEY => '', ENDKEY => '1'} 2023-03-27 03:54:13,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles c0982fa95f3b2e90e5d67d21d9f32707 2023-03-27 03:54:13,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:13,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c0982fa95f3b2e90e5d67d21d9f32707 2023-03-27 03:54:13,070 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c0982fa95f3b2e90e5d67d21d9f32707 2023-03-27 03:54:13,072 INFO [StoreOpener-c0982fa95f3b2e90e5d67d21d9f32707-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c0982fa95f3b2e90e5d67d21d9f32707 2023-03-27 03:54:13,073 DEBUG [StoreOpener-c0982fa95f3b2e90e5d67d21d9f32707-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/c0982fa95f3b2e90e5d67d21d9f32707/family1 2023-03-27 03:54:13,073 DEBUG [StoreOpener-c0982fa95f3b2e90e5d67d21d9f32707-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/c0982fa95f3b2e90e5d67d21d9f32707/family1 2023-03-27 03:54:13,074 INFO [StoreOpener-c0982fa95f3b2e90e5d67d21d9f32707-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c0982fa95f3b2e90e5d67d21d9f32707 columnFamilyName family1 2023-03-27 03:54:13,075 INFO [StoreOpener-c0982fa95f3b2e90e5d67d21d9f32707-1] regionserver.HStore(310): Store=c0982fa95f3b2e90e5d67d21d9f32707/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:54:13,076 INFO [StoreOpener-c0982fa95f3b2e90e5d67d21d9f32707-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region c0982fa95f3b2e90e5d67d21d9f32707 2023-03-27 03:54:13,077 DEBUG [StoreOpener-c0982fa95f3b2e90e5d67d21d9f32707-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/c0982fa95f3b2e90e5d67d21d9f32707/family2 2023-03-27 03:54:13,077 DEBUG [StoreOpener-c0982fa95f3b2e90e5d67d21d9f32707-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/c0982fa95f3b2e90e5d67d21d9f32707/family2 2023-03-27 03:54:13,078 INFO [StoreOpener-c0982fa95f3b2e90e5d67d21d9f32707-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c0982fa95f3b2e90e5d67d21d9f32707 columnFamilyName family2 2023-03-27 03:54:13,079 INFO [StoreOpener-c0982fa95f3b2e90e5d67d21d9f32707-1] regionserver.HStore(310): Store=c0982fa95f3b2e90e5d67d21d9f32707/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:54:13,080 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/c0982fa95f3b2e90e5d67d21d9f32707 2023-03-27 03:54:13,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/c0982fa95f3b2e90e5d67d21d9f32707 2023-03-27 03:54:13,083 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-27 03:54:13,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c0982fa95f3b2e90e5d67d21d9f32707 2023-03-27 03:54:13,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/c0982fa95f3b2e90e5d67d21d9f32707/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:54:13,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c0982fa95f3b2e90e5d67d21d9f32707; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66485339, jitterRate=-0.00929124653339386}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-27 03:54:13,087 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c0982fa95f3b2e90e5d67d21d9f32707: 2023-03-27 03:54:13,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707., pid=31, masterSystemTime=1679889253066 2023-03-27 03:54:13,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707. 2023-03-27 03:54:13,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707. 2023-03-27 03:54:13,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6. 2023-03-27 03:54:13,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f2819f01ee4f081befffef519bfab5a6, NAME => 'testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6.', STARTKEY => '2', ENDKEY => ''} 2023-03-27 03:54:13,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles f2819f01ee4f081befffef519bfab5a6 2023-03-27 03:54:13,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:13,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f2819f01ee4f081befffef519bfab5a6 2023-03-27 03:54:13,090 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=c0982fa95f3b2e90e5d67d21d9f32707, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:13,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f2819f01ee4f081befffef519bfab5a6 2023-03-27 03:54:13,090 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679889253090"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889253090"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889253090"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889253090"}]},"ts":"1679889253090"} 2023-03-27 03:54:13,092 INFO [StoreOpener-f2819f01ee4f081befffef519bfab5a6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f2819f01ee4f081befffef519bfab5a6 2023-03-27 03:54:13,094 DEBUG [StoreOpener-f2819f01ee4f081befffef519bfab5a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1 2023-03-27 03:54:13,094 DEBUG [StoreOpener-f2819f01ee4f081befffef519bfab5a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1 2023-03-27 03:54:13,095 INFO [StoreOpener-f2819f01ee4f081befffef519bfab5a6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f2819f01ee4f081befffef519bfab5a6 columnFamilyName family1 2023-03-27 03:54:13,095 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-03-27 03:54:13,095 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; OpenRegionProcedure c0982fa95f3b2e90e5d67d21d9f32707, server=jenkins-hbase4.apache.org,39727,1679889218560 in 179 msec 2023-03-27 03:54:13,096 INFO [StoreOpener-f2819f01ee4f081befffef519bfab5a6-1] regionserver.HStore(310): Store=f2819f01ee4f081befffef519bfab5a6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:54:13,096 INFO [StoreOpener-f2819f01ee4f081befffef519bfab5a6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region f2819f01ee4f081befffef519bfab5a6 2023-03-27 03:54:13,098 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=c0982fa95f3b2e90e5d67d21d9f32707, ASSIGN in 340 msec 2023-03-27 03:54:13,099 DEBUG [StoreOpener-f2819f01ee4f081befffef519bfab5a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family2 2023-03-27 03:54:13,099 DEBUG [StoreOpener-f2819f01ee4f081befffef519bfab5a6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family2 2023-03-27 03:54:13,100 INFO [StoreOpener-f2819f01ee4f081befffef519bfab5a6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f2819f01ee4f081befffef519bfab5a6 columnFamilyName family2 2023-03-27 03:54:13,101 INFO [StoreOpener-f2819f01ee4f081befffef519bfab5a6-1] regionserver.HStore(310): Store=f2819f01ee4f081befffef519bfab5a6/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:54:13,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6 2023-03-27 03:54:13,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6 2023-03-27 03:54:13,105 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-27 03:54:13,106 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f2819f01ee4f081befffef519bfab5a6 2023-03-27 03:54:13,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:54:13,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f2819f01ee4f081befffef519bfab5a6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60217896, jitterRate=-0.102683424949646}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-27 03:54:13,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f2819f01ee4f081befffef519bfab5a6: 2023-03-27 03:54:13,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6., pid=32, masterSystemTime=1679889253066 2023-03-27 03:54:13,111 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6. 2023-03-27 03:54:13,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6. 2023-03-27 03:54:13,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1. 2023-03-27 03:54:13,111 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1002361bf1e6585024f86241020836f1, NAME => 'testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1.', STARTKEY => '1', ENDKEY => '2'} 2023-03-27 03:54:13,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 1002361bf1e6585024f86241020836f1 2023-03-27 03:54:13,112 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=f2819f01ee4f081befffef519bfab5a6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:13,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:13,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1002361bf1e6585024f86241020836f1 2023-03-27 03:54:13,112 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1679889253112"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889253112"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889253112"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889253112"}]},"ts":"1679889253112"} 2023-03-27 03:54:13,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1002361bf1e6585024f86241020836f1 2023-03-27 03:54:13,114 INFO [StoreOpener-1002361bf1e6585024f86241020836f1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1002361bf1e6585024f86241020836f1 2023-03-27 03:54:13,115 DEBUG [StoreOpener-1002361bf1e6585024f86241020836f1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/1002361bf1e6585024f86241020836f1/family1 2023-03-27 03:54:13,115 DEBUG [StoreOpener-1002361bf1e6585024f86241020836f1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/1002361bf1e6585024f86241020836f1/family1 2023-03-27 03:54:13,116 INFO [StoreOpener-1002361bf1e6585024f86241020836f1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1002361bf1e6585024f86241020836f1 columnFamilyName family1 2023-03-27 03:54:13,116 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=30 2023-03-27 03:54:13,116 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=30, state=SUCCESS; OpenRegionProcedure f2819f01ee4f081befffef519bfab5a6, server=jenkins-hbase4.apache.org,39727,1679889218560 in 200 msec 2023-03-27 03:54:13,117 INFO [StoreOpener-1002361bf1e6585024f86241020836f1-1] regionserver.HStore(310): Store=1002361bf1e6585024f86241020836f1/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:54:13,117 INFO [StoreOpener-1002361bf1e6585024f86241020836f1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 1002361bf1e6585024f86241020836f1 2023-03-27 03:54:13,118 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=f2819f01ee4f081befffef519bfab5a6, ASSIGN in 361 msec 2023-03-27 03:54:13,119 DEBUG [StoreOpener-1002361bf1e6585024f86241020836f1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/1002361bf1e6585024f86241020836f1/family2 2023-03-27 03:54:13,119 DEBUG [StoreOpener-1002361bf1e6585024f86241020836f1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/1002361bf1e6585024f86241020836f1/family2 2023-03-27 03:54:13,119 INFO [StoreOpener-1002361bf1e6585024f86241020836f1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1002361bf1e6585024f86241020836f1 columnFamilyName family2 2023-03-27 03:54:13,120 INFO [StoreOpener-1002361bf1e6585024f86241020836f1-1] regionserver.HStore(310): Store=1002361bf1e6585024f86241020836f1/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:54:13,121 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/1002361bf1e6585024f86241020836f1 2023-03-27 03:54:13,121 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/1002361bf1e6585024f86241020836f1 2023-03-27 03:54:13,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-27 03:54:13,125 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1002361bf1e6585024f86241020836f1 2023-03-27 03:54:13,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/1002361bf1e6585024f86241020836f1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:54:13,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1002361bf1e6585024f86241020836f1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72484891, jitterRate=0.08010904490947723}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-27 03:54:13,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1002361bf1e6585024f86241020836f1: 2023-03-27 03:54:13,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1., pid=33, masterSystemTime=1679889253066 2023-03-27 03:54:13,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1. 2023-03-27 03:54:13,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1. 2023-03-27 03:54:13,133 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=1002361bf1e6585024f86241020836f1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:13,133 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1679889253133"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889253133"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889253133"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889253133"}]},"ts":"1679889253133"} 2023-03-27 03:54:13,137 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=29 2023-03-27 03:54:13,137 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=29, state=SUCCESS; OpenRegionProcedure 1002361bf1e6585024f86241020836f1, server=jenkins-hbase4.apache.org,39727,1679889218560 in 219 msec 2023-03-27 03:54:13,139 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=27 2023-03-27 03:54:13,139 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=1002361bf1e6585024f86241020836f1, ASSIGN in 382 msec 2023-03-27 03:54:13,140 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-27 03:54:13,140 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889253140"}]},"ts":"1679889253140"} 2023-03-27 03:54:13,142 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-03-27 03:54:13,145 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-03-27 03:54:13,147 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 1.2530 sec 2023-03-27 03:54:14,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-03-27 03:54:14,003 INFO [Listener at localhost/36131] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-03-27 03:54:14,011 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:14,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,014 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-27 03:54:14,015 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:14,015 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-27 03:54:14,015 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-27 03:54:14,015 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,015 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,018 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:14,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:14,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:14,018 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:14,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,019 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:14,019 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:14,019 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:14,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,020 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,020 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,021 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-27 03:54:14,021 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:14,021 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-27 03:54:14,021 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6. 2023-03-27 03:54:14,021 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6. started... 2023-03-27 03:54:14,028 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f2819f01ee4f081befffef519bfab5a6 2/2 column families, dataSize=48 B heapSize=632 B 2023-03-27 03:54:14,029 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1. 2023-03-27 03:54:14,029 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1. started... 2023-03-27 03:54:14,032 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-27 03:54:14,041 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:14,041 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707. 2023-03-27 03:54:14,041 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 1002361bf1e6585024f86241020836f1: 2023-03-27 03:54:14,042 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1. 2023-03-27 03:54:14,042 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-27 03:54:14,042 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707. started... 2023-03-27 03:54:14,043 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for c0982fa95f3b2e90e5d67d21d9f32707: 2023-03-27 03:54:14,043 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707. 2023-03-27 03:54:14,043 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-27 03:54:14,088 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230327bacc767b7fcb41e1b88578dfc809fa5a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230327bacc767b7fcb41e1b88578dfc809fa5a 2023-03-27 03:54:14,539 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/.tmp/family1/331922d4dec649a784daeb375f60e6c4 2023-03-27 03:54:14,548 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/.tmp/family1/331922d4dec649a784daeb375f60e6c4 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1/331922d4dec649a784daeb375f60e6c4 2023-03-27 03:54:14,556 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1/331922d4dec649a784daeb375f60e6c4, entries=1, sequenceid=5, filesize=4.9 K 2023-03-27 03:54:14,557 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for f2819f01ee4f081befffef519bfab5a6 in 529ms, sequenceid=5, compaction requested=false 2023-03-27 03:54:14,558 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-03-27 03:54:14,558 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f2819f01ee4f081befffef519bfab5a6: 2023-03-27 03:54:14,558 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6. 2023-03-27 03:54:14,558 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-27 03:54:14,558 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-27 03:54:14,558 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:14,559 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-27 03:54:14,559 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-27 03:54:14,560 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:14,560 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:14,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:14,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:14,561 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,561 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:14,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:14,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:14,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:14,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:14,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-27 03:54:14,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6fb5cdaa[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:14,563 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-27 03:54:14,563 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,564 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,564 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-27 03:54:14,564 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-27 03:54:14,564 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:14,565 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:14,565 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:14,568 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:14,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:14,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:14,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:14,569 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-27 03:54:14,569 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:14,569 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-27 03:54:14,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:14,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:14,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:14,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:14,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:14,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:14,575 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-27 03:54:14,575 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-27 03:54:14,575 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:14,575 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:14,575 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,575 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:14,577 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,577 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:14,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,577 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:14,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:14,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:14,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:14,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:14,579 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:14,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:14,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:14,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:14,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:14,583 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:14,588 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:14,588 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,588 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,588 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:14,588 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-27 03:54:14,588 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:14,588 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:14,588 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:14,588 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,588 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,588 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:14,588 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,588 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:14,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:14,589 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:14,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:14,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:14,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:14,689 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:14,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:14,705 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:14,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:14,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,708 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-03-27 03:54:14,708 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:14,708 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-03-27 03:54:14,708 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-03-27 03:54:14,709 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,709 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,711 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:14,711 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:14,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:14,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:14,711 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:14,712 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:14,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,712 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:14,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,713 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,720 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-03-27 03:54:14,720 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:14,720 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-03-27 03:54:14,720 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:14,720 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-03-27 03:54:14,721 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6. 2023-03-27 03:54:14,721 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6. started... 2023-03-27 03:54:14,721 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f2819f01ee4f081befffef519bfab5a6 2/2 column families, dataSize=90 B heapSize=752 B 2023-03-27 03:54:14,721 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1. 2023-03-27 03:54:14,722 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-27 03:54:14,722 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:14,722 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707. 2023-03-27 03:54:14,722 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1. started... 2023-03-27 03:54:14,722 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707. started... 2023-03-27 03:54:14,722 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 1002361bf1e6585024f86241020836f1: 2023-03-27 03:54:14,722 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1. 2023-03-27 03:54:14,722 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for c0982fa95f3b2e90e5d67d21d9f32707: 2023-03-27 03:54:14,722 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-27 03:54:14,722 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707. 2023-03-27 03:54:14,722 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-27 03:54:15,147 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230327d6b08efbf09a476dbaca74cd6263429c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230327d6b08efbf09a476dbaca74cd6263429c 2023-03-27 03:54:15,555 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/.tmp/family1/cf126f0995e240b1a4c5ca9959cd5655 2023-03-27 03:54:15,561 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/.tmp/family1/cf126f0995e240b1a4c5ca9959cd5655 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1/cf126f0995e240b1a4c5ca9959cd5655 2023-03-27 03:54:15,565 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1/cf126f0995e240b1a4c5ca9959cd5655, entries=2, sequenceid=9, filesize=5.0 K 2023-03-27 03:54:15,566 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for f2819f01ee4f081befffef519bfab5a6 in 845ms, sequenceid=9, compaction requested=false 2023-03-27 03:54:15,567 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f2819f01ee4f081befffef519bfab5a6: 2023-03-27 03:54:15,567 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6. 2023-03-27 03:54:15,567 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-27 03:54:15,567 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-27 03:54:15,567 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:15,567 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-03-27 03:54:15,567 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-03-27 03:54:15,569 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:15,569 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:15,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:15,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:15,569 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,569 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:15,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:15,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:15,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:15,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:15,571 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-03-27 03:54:15,571 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@448e2fd7[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:15,571 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-03-27 03:54:15,571 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,572 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,572 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-03-27 03:54:15,572 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-03-27 03:54:15,572 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:15,572 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:15,572 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:15,574 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:15,574 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-03-27 03:54:15,574 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:15,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:15,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:15,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:15,574 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-03-27 03:54:15,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:15,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:15,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:15,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:15,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:15,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:15,578 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-03-27 03:54:15,578 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-03-27 03:54:15,578 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:15,578 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:15,578 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,578 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:15,581 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,581 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:15,581 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:15,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:15,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:15,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:15,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:15,582 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:15,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:15,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,583 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:15,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:15,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:15,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:15,586 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:15,586 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:15,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:15,586 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:15,586 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,586 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,586 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:15,586 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,586 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,586 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,586 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:15,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-03-27 03:54:15,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:15,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:15,587 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:15,587 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:15,587 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-03-27 03:54:15,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:15,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:15,687 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:15,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:15,696 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38271] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-03-27 03:54:15,698 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-27 03:54:15,699 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-27 03:54:15,700 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679889855700 2023-03-27 03:54:15,700 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:54:15,701 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-27 03:54:15,701 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-03-27 03:54:15,701 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-27 03:54:15,701 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] zookeeper.ReadOnlyZKClient(139): Connect 0x1ced8ffe to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:54:15,706 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@34d68d05, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:54:15,706 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230327 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-27 03:54:15,768 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303276a5b62bd82c745809f5e0e7072dee891 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202303276a5b62bd82c745809f5e0e7072dee891 2023-03-27 03:54:15,770 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230327 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-27 03:54:15,785 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-27 03:54:15,786 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36060, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-27 03:54:15,813 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-27 03:54:15,817 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49616, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-27 03:54:15,845 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230327/family1/563054d4c4414b24b5c0ab3b9a96e243 first=Optional[k0] last=Optional[k1] 2023-03-27 03:54:15,846 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230327/family1/563054d4c4414b24b5c0ab3b9a96e243}] 2023-03-27 03:54:15,857 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230327/family1/563054d4c4414b24b5c0ab3b9a96e243 for inclusion in f2819f01ee4f081befffef519bfab5a6/family1 2023-03-27 03:54:15,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-03-27 03:54:15,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-27 03:54:15,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for f2819f01ee4f081befffef519bfab5a6: 2023-03-27 03:54:15,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230327/family1/563054d4c4414b24b5c0ab3b9a96e243 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testScannerOnBulkLoadRefHFiles__8rb7b4utq01vfk5gv1ie2rl141plfdpnh5bub0l1tem364dbmuat7542g8b0m03b/family1/563054d4c4414b24b5c0ab3b9a96e243 2023-03-27 03:54:15,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testScannerOnBulkLoadRefHFiles__8rb7b4utq01vfk5gv1ie2rl141plfdpnh5bub0l1tem364dbmuat7542g8b0m03b/family1/563054d4c4414b24b5c0ab3b9a96e243 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1/a4745676404947af9cccd8aca0901262_SeqId_12_ 2023-03-27 03:54:15,869 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testScannerOnBulkLoadRefHFiles__8rb7b4utq01vfk5gv1ie2rl141plfdpnh5bub0l1tem364dbmuat7542g8b0m03b/family1/563054d4c4414b24b5c0ab3b9a96e243 into f2819f01ee4f081befffef519bfab5a6/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1/a4745676404947af9cccd8aca0901262_SeqId_12_ - updating store file list. 2023-03-27 03:54:15,874 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1/a4745676404947af9cccd8aca0901262_SeqId_12_ into f2819f01ee4f081befffef519bfab5a6/family1 2023-03-27 03:54:15,874 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testScannerOnBulkLoadRefHFiles__8rb7b4utq01vfk5gv1ie2rl141plfdpnh5bub0l1tem364dbmuat7542g8b0m03b/family1/563054d4c4414b24b5c0ab3b9a96e243 into f2819f01ee4f081befffef519bfab5a6/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1/a4745676404947af9cccd8aca0901262_SeqId_12_) 2023-03-27 03:54:15,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testScannerOnBulkLoadRefHFiles__8rb7b4utq01vfk5gv1ie2rl141plfdpnh5bub0l1tem364dbmuat7542g8b0m03b/family1/563054d4c4414b24b5c0ab3b9a96e243 2023-03-27 03:54:15,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-27 03:54:15,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-27 03:54:15,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(1912): f2819f01ee4f081befffef519bfab5a6/family1 is initiating minor compaction (all files) 2023-03-27 03:54:15,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@ef65c37; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:15,876 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f2819f01ee4f081befffef519bfab5a6/family1 in testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6. 2023-03-27 03:54:15,876 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1/331922d4dec649a784daeb375f60e6c4, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1/cf126f0995e240b1a4c5ca9959cd5655, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1/a4745676404947af9cccd8aca0901262_SeqId_12_] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/.tmp, totalSize=15.1 K 2023-03-27 03:54:15,877 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 331922d4dec649a784daeb375f60e6c4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889254003 2023-03-27 03:54:15,877 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting cf126f0995e240b1a4c5ca9959cd5655, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889254003 2023-03-27 03:54:15,878 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting a4745676404947af9cccd8aca0901262_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889254003 2023-03-27 03:54:15,885 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230327 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-27 03:54:15,886 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:15,889 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230327bacc767b7fcb41e1b88578dfc809fa5a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230327bacc767b7fcb41e1b88578dfc809fa5a 2023-03-27 03:54:15,891 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230327d6b08efbf09a476dbaca74cd6263429c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230327d6b08efbf09a476dbaca74cd6263429c 2023-03-27 03:54:15,891 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-03-27 03:54:15,891 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230327' 2023-03-27 03:54:15,892 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230327 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-03-27 03:54:15,892 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-03-27 03:54:15,892 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:54:15,894 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-03-27 03:54:15,894 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] throttle.PressureAwareThroughputController(145): f2819f01ee4f081befffef519bfab5a6#family1#compaction#38 average throughput is 0.03 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:54:15,895 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 197 msec 2023-03-27 03:54:15,933 INFO [Listener at localhost/36131] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=253 (was 256), OpenFileDescriptor=597 (was 599), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=450 (was 445) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=8876 (was 8909) 2023-03-27 03:54:15,946 INFO [Listener at localhost/36131] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=253, OpenFileDescriptor=597, MaxFileDescriptor=60000, SystemLoadAverage=450, ProcessCount=170, AvailableMemoryMB=8875 2023-03-27 03:54:15,949 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-27 03:54:15,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:15,951 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-27 03:54:15,951 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-03-27 03:54:15,952 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-27 03:54:15,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-27 03:54:15,954 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821 2023-03-27 03:54:15,954 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821 empty. 2023-03-27 03:54:15,955 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821 2023-03-27 03:54:15,955 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-03-27 03:54:15,966 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-27 03:54:15,968 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => a7921456431e437526c5b87521696821, NAME => 'testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp 2023-03-27 03:54:16,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-27 03:54:16,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-27 03:54:16,375 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:16,375 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing a7921456431e437526c5b87521696821, disabling compactions & flushes 2023-03-27 03:54:16,375 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:16,375 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:16,375 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. after waiting 0 ms 2023-03-27 03:54:16,376 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:16,376 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:16,376 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for a7921456431e437526c5b87521696821: 2023-03-27 03:54:16,378 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-27 03:54:16,380 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679889256380"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889256380"}]},"ts":"1679889256380"} 2023-03-27 03:54:16,382 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-27 03:54:16,383 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-27 03:54:16,383 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889256383"}]},"ts":"1679889256383"} 2023-03-27 03:54:16,384 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-03-27 03:54:16,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=a7921456431e437526c5b87521696821, ASSIGN}] 2023-03-27 03:54:16,390 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=a7921456431e437526c5b87521696821, ASSIGN 2023-03-27 03:54:16,393 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=a7921456431e437526c5b87521696821, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=false 2023-03-27 03:54:16,544 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=a7921456431e437526c5b87521696821, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:16,544 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679889256544"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889256544"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889256544"}]},"ts":"1679889256544"} 2023-03-27 03:54:16,546 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure a7921456431e437526c5b87521696821, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:54:16,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-27 03:54:16,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:16,701 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a7921456431e437526c5b87521696821, NAME => 'testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821.', STARTKEY => '', ENDKEY => ''} 2023-03-27 03:54:16,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy a7921456431e437526c5b87521696821 2023-03-27 03:54:16,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:16,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a7921456431e437526c5b87521696821 2023-03-27 03:54:16,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a7921456431e437526c5b87521696821 2023-03-27 03:54:16,703 INFO [StoreOpener-a7921456431e437526c5b87521696821-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a7921456431e437526c5b87521696821 2023-03-27 03:54:16,705 DEBUG [StoreOpener-a7921456431e437526c5b87521696821-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1 2023-03-27 03:54:16,705 DEBUG [StoreOpener-a7921456431e437526c5b87521696821-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1 2023-03-27 03:54:16,705 INFO [StoreOpener-a7921456431e437526c5b87521696821-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a7921456431e437526c5b87521696821 columnFamilyName family1 2023-03-27 03:54:16,707 INFO [StoreOpener-a7921456431e437526c5b87521696821-1] regionserver.HStore(310): Store=a7921456431e437526c5b87521696821/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:54:16,708 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821 2023-03-27 03:54:16,708 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821 2023-03-27 03:54:16,712 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a7921456431e437526c5b87521696821 2023-03-27 03:54:16,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:54:16,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a7921456431e437526c5b87521696821; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66906319, jitterRate=-0.0030181556940078735}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-27 03:54:16,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a7921456431e437526c5b87521696821: 2023-03-27 03:54:16,717 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821., pid=37, masterSystemTime=1679889256698 2023-03-27 03:54:16,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:16,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:16,719 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=a7921456431e437526c5b87521696821, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:16,719 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1679889256719"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889256719"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889256719"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889256719"}]},"ts":"1679889256719"} 2023-03-27 03:54:16,722 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-03-27 03:54:16,722 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure a7921456431e437526c5b87521696821, server=jenkins-hbase4.apache.org,39727,1679889218560 in 175 msec 2023-03-27 03:54:16,725 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-03-27 03:54:16,725 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=a7921456431e437526c5b87521696821, ASSIGN in 333 msec 2023-03-27 03:54:16,727 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-27 03:54:16,727 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889256727"}]},"ts":"1679889256727"} 2023-03-27 03:54:16,728 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-03-27 03:54:16,731 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-27 03:54:16,731 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/.tmp/family1/19b6ea841bfb429c896d31ef61c5f51f as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1/19b6ea841bfb429c896d31ef61c5f51f 2023-03-27 03:54:16,733 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 782 msec 2023-03-27 03:54:16,740 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in f2819f01ee4f081befffef519bfab5a6/family1 of f2819f01ee4f081befffef519bfab5a6 into 19b6ea841bfb429c896d31ef61c5f51f(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:54:16,740 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f2819f01ee4f081befffef519bfab5a6: 2023-03-27 03:54:16,740 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1679889255876; duration=0sec 2023-03-27 03:54:16,740 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:16,797 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1/331922d4dec649a784daeb375f60e6c4, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1/cf126f0995e240b1a4c5ca9959cd5655, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1/a4745676404947af9cccd8aca0901262_SeqId_12_] to archive 2023-03-27 03:54:16,798 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:16,800 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1/331922d4dec649a784daeb375f60e6c4 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1/331922d4dec649a784daeb375f60e6c4 2023-03-27 03:54:16,802 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1/cf126f0995e240b1a4c5ca9959cd5655 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1/cf126f0995e240b1a4c5ca9959cd5655 2023-03-27 03:54:16,803 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1/a4745676404947af9cccd8aca0901262_SeqId_12_ to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/family1/a4745676404947af9cccd8aca0901262_SeqId_12_ 2023-03-27 03:54:16,864 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-03-27 03:54:17,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-03-27 03:54:17,058 INFO [Listener at localhost/36131] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-03-27 03:54:17,095 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:17,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,112 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-27 03:54:17,112 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:17,112 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-27 03:54:17,112 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-27 03:54:17,113 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,113 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,114 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:17,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:17,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:17,114 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,115 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,115 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:17,115 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:17,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,115 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,132 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,132 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-27 03:54:17,132 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:17,132 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-27 03:54:17,132 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:17,133 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:17,133 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:17,133 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. started... 2023-03-27 03:54:17,145 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a7921456431e437526c5b87521696821 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-27 03:54:17,201 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130cc7f3c46eeb045f1890ca8f502d446c0 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130cc7f3c46eeb045f1890ca8f502d446c0 2023-03-27 03:54:17,206 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/071a858929244c46a803bdda5f31810d 2023-03-27 03:54:17,212 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/071a858929244c46a803bdda5f31810d as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/071a858929244c46a803bdda5f31810d 2023-03-27 03:54:17,219 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/071a858929244c46a803bdda5f31810d, entries=1000, sequenceid=5, filesize=165.1 K 2023-03-27 03:54:17,220 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for a7921456431e437526c5b87521696821 in 76ms, sequenceid=5, compaction requested=false 2023-03-27 03:54:17,221 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a7921456431e437526c5b87521696821: 2023-03-27 03:54:17,221 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:17,221 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:17,221 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:17,221 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,221 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-27 03:54:17,221 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-27 03:54:17,223 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,223 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:17,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:17,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:17,224 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,224 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:17,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:17,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:17,225 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-27 03:54:17,225 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54c09864[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:17,225 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-27 03:54:17,225 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,226 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,227 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-27 03:54:17,227 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-27 03:54:17,227 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:17,227 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,227 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:17,229 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-27 03:54:17,229 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:17,229 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-27 03:54:17,229 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:17,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:17,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:17,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:17,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:17,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:17,233 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:54:17,233 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-27 03:54:17,233 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:17,233 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:17,233 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,234 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:17,235 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,235 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:17,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:17,235 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:17,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:17,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:17,236 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:17,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:17,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,238 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:17,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,241 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:17,241 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:17,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:17,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:17,241 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,241 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,241 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,241 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,241 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,241 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,241 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:17,241 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-27 03:54:17,241 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,241 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-27 03:54:17,242 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:17,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:17,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:17,242 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:17,242 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:17,342 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:17,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:17,349 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:17,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:17,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,351 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-27 03:54:17,351 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:17,351 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-27 03:54:17,351 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-27 03:54:17,351 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,352 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,353 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:17,353 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:17,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:17,353 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,353 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:17,354 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:17,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,354 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,354 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-27 03:54:17,354 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,355 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-27 03:54:17,355 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:17,355 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-27 03:54:17,355 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:17,355 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:17,356 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:17,356 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. started... 2023-03-27 03:54:17,356 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a7921456431e437526c5b87521696821 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-27 03:54:17,391 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128f4fda35da1fe49d3a436c5fc2e0cf4e6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151128f4fda35da1fe49d3a436c5fc2e0cf4e6 2023-03-27 03:54:17,416 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/884f448a7ab54f44848b58c9b3fe3890 2023-03-27 03:54:17,422 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/884f448a7ab54f44848b58c9b3fe3890 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/884f448a7ab54f44848b58c9b3fe3890 2023-03-27 03:54:17,428 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/884f448a7ab54f44848b58c9b3fe3890, entries=1, sequenceid=9, filesize=5.0 K 2023-03-27 03:54:17,429 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for a7921456431e437526c5b87521696821 in 73ms, sequenceid=9, compaction requested=false 2023-03-27 03:54:17,429 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a7921456431e437526c5b87521696821: 2023-03-27 03:54:17,429 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:17,429 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:17,429 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:17,430 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,430 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-27 03:54:17,430 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-27 03:54:17,436 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,437 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:17,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:17,437 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,437 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:17,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:17,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:17,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:17,439 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-27 03:54:17,439 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@784e539a[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:17,439 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-27 03:54:17,439 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,440 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,441 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-27 03:54:17,441 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-27 03:54:17,441 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:17,441 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,441 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:17,445 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-27 03:54:17,445 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:17,445 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-27 03:54:17,445 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:17,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:17,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:17,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:17,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:17,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:17,449 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:54:17,449 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-27 03:54:17,449 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:17,449 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:17,449 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,449 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:17,451 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,451 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,452 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:17,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:17,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:17,452 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:17,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:17,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:17,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:17,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,454 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:17,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,456 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:17,456 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,456 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:17,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:17,456 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,456 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:17,456 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,457 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,456 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-27 03:54:17,457 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:17,457 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,457 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,457 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,457 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-27 03:54:17,457 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:17,457 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:17,459 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:17,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:17,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:17,557 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:17,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:17,563 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:17,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:17,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,566 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-27 03:54:17,566 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:17,566 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-27 03:54:17,566 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-27 03:54:17,567 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,567 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,568 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:17,568 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:17,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:17,568 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,568 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:17,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,569 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:17,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,569 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,570 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-27 03:54:17,570 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,570 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-27 03:54:17,570 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:17,570 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-27 03:54:17,570 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:17,570 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:17,570 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:17,570 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. started... 2023-03-27 03:54:17,571 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a7921456431e437526c5b87521696821 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:17,588 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512010afead3c5846488b87b93f3473c8aba0 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512010afead3c5846488b87b93f3473c8aba0 2023-03-27 03:54:17,596 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/c4fa0e28d8f34a1b87cca9de0f92bb40 2023-03-27 03:54:17,602 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/c4fa0e28d8f34a1b87cca9de0f92bb40 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/c4fa0e28d8f34a1b87cca9de0f92bb40 2023-03-27 03:54:17,606 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/c4fa0e28d8f34a1b87cca9de0f92bb40, entries=1, sequenceid=13, filesize=4.9 K 2023-03-27 03:54:17,607 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a7921456431e437526c5b87521696821 in 36ms, sequenceid=13, compaction requested=true 2023-03-27 03:54:17,608 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a7921456431e437526c5b87521696821: 2023-03-27 03:54:17,608 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:17,608 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:17,608 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:17,608 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,608 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-27 03:54:17,608 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-27 03:54:17,609 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,610 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:17,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:17,610 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:17,610 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:17,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:17,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:17,612 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-27 03:54:17,612 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@39d709a4[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:17,612 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-27 03:54:17,612 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,614 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,614 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-27 03:54:17,614 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-27 03:54:17,614 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:17,614 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,614 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:17,616 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,616 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-27 03:54:17,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:17,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:17,616 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:17,616 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-27 03:54:17,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:17,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:17,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:17,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:17,619 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:54:17,619 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-27 03:54:17,619 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:17,619 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:17,619 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,619 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:17,621 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,621 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,621 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:17,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:17,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:17,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,622 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,623 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:17,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:17,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:17,626 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,626 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:17,626 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,626 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:17,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:17,626 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-27 03:54:17,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:17,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:17,626 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,626 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,626 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:17,626 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,626 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:17,627 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:17,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:17,627 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:17,627 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:17,627 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:17,628 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-27 03:54:17,727 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:17,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:17,735 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:17,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:17,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,737 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-27 03:54:17,737 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:17,737 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-27 03:54:17,737 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-27 03:54:17,738 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,738 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,740 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:17,740 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:17,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:17,740 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,740 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:17,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,741 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:17,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,741 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,741 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-27 03:54:17,741 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,741 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-27 03:54:17,741 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:17,741 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-27 03:54:17,743 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:17,743 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:17,743 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:17,743 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. started... 2023-03-27 03:54:17,743 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a7921456431e437526c5b87521696821 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:17,763 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205d63c758208174e5b94b25ef8efb746d6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205d63c758208174e5b94b25ef8efb746d6 2023-03-27 03:54:17,778 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/c8f53d60329a475fac430613588f427c 2023-03-27 03:54:17,790 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/c8f53d60329a475fac430613588f427c as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/c8f53d60329a475fac430613588f427c 2023-03-27 03:54:17,796 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/c8f53d60329a475fac430613588f427c, entries=1, sequenceid=17, filesize=4.9 K 2023-03-27 03:54:17,797 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a7921456431e437526c5b87521696821 in 54ms, sequenceid=17, compaction requested=true 2023-03-27 03:54:17,797 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a7921456431e437526c5b87521696821: 2023-03-27 03:54:17,797 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:17,797 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:17,797 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:17,797 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,797 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-27 03:54:17,797 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-27 03:54:17,799 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,799 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:17,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:17,799 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,799 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:17,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:17,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:17,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:17,800 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-27 03:54:17,800 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c1279de[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:17,800 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-27 03:54:17,801 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,802 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,802 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-27 03:54:17,802 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-27 03:54:17,802 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,802 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:17,802 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:17,803 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,803 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-27 03:54:17,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,803 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:17,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:17,803 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-27 03:54:17,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:17,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:17,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:17,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:17,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:17,806 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:54:17,806 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-27 03:54:17,806 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:17,806 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:17,806 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,806 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:17,809 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,809 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:17,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:17,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,809 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:17,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:17,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:17,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:17,810 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:17,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,811 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:17,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,814 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,814 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:17,814 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,814 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:17,814 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:17,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:17,814 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-27 03:54:17,814 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:17,814 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,815 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,815 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,815 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,815 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:17,815 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:17,815 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:17,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:17,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:17,915 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:17,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:17,921 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:17,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:17,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,922 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-27 03:54:17,922 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:17,923 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-27 03:54:17,923 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-27 03:54:17,923 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,923 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,924 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,924 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:17,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:17,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:17,925 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,925 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:17,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,925 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:17,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,926 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,926 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-27 03:54:17,926 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,926 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-27 03:54:17,926 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:17,926 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-27 03:54:17,926 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:17,926 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:17,926 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:17,927 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. started... 2023-03-27 03:54:17,927 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a7921456431e437526c5b87521696821 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:17,939 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228d319b6ebb33a4444b3db74f0c1749a2c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228d319b6ebb33a4444b3db74f0c1749a2c 2023-03-27 03:54:17,944 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/73c1a4cb91684d718f1d42c87264d0d8 2023-03-27 03:54:17,949 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/73c1a4cb91684d718f1d42c87264d0d8 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/73c1a4cb91684d718f1d42c87264d0d8 2023-03-27 03:54:17,953 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/73c1a4cb91684d718f1d42c87264d0d8, entries=1, sequenceid=21, filesize=4.9 K 2023-03-27 03:54:17,954 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a7921456431e437526c5b87521696821 in 27ms, sequenceid=21, compaction requested=true 2023-03-27 03:54:17,954 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a7921456431e437526c5b87521696821: 2023-03-27 03:54:17,954 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:17,954 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:17,954 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:17,954 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,954 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-27 03:54:17,954 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-27 03:54:17,960 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,960 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:17,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:17,961 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,961 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:17,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:17,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:17,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:17,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-27 03:54:17,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c41d380[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:17,962 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-27 03:54:17,963 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,964 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,964 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-27 03:54:17,964 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-27 03:54:17,964 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:17,964 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,964 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:17,965 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,966 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-27 03:54:17,966 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:17,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:17,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:17,966 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-27 03:54:17,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:17,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:17,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:17,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:17,973 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:54:17,973 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-27 03:54:17,973 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:17,973 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:17,973 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,973 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:17,974 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,975 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:17,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:17,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:17,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,979 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:17,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:17,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:17,983 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:17,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,985 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:17,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,987 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,987 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,988 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,988 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:17,988 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,988 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,988 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:17,988 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:17,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:17,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:17,988 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:17,988 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:17,988 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-27 03:54:17,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:17,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:17,988 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:17,988 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:17,989 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-27 03:54:17,989 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:18,089 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:18,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:18,095 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:18,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:18,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,097 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-27 03:54:18,097 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:18,097 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-27 03:54:18,097 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-27 03:54:18,099 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,099 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,100 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,100 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:18,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:18,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:18,101 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,101 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:18,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,102 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:18,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,102 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,102 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-27 03:54:18,102 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,102 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-27 03:54:18,102 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:18,102 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-27 03:54:18,103 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:18,103 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:18,103 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:18,103 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. started... 2023-03-27 03:54:18,103 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a7921456431e437526c5b87521696821 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:18,143 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231991479bb45e347d0a6d84a7203e16c1c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231991479bb45e347d0a6d84a7203e16c1c 2023-03-27 03:54:18,162 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/f61d186cbc5d4c2cad8df022e7a6dd35 2023-03-27 03:54:18,181 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/f61d186cbc5d4c2cad8df022e7a6dd35 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/f61d186cbc5d4c2cad8df022e7a6dd35 2023-03-27 03:54:18,189 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/f61d186cbc5d4c2cad8df022e7a6dd35, entries=1, sequenceid=25, filesize=4.9 K 2023-03-27 03:54:18,190 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a7921456431e437526c5b87521696821 in 87ms, sequenceid=25, compaction requested=true 2023-03-27 03:54:18,190 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a7921456431e437526c5b87521696821: 2023-03-27 03:54:18,190 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:18,190 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:18,191 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:18,191 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,191 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-27 03:54:18,191 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-27 03:54:18,194 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:18,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:18,194 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:18,195 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,195 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:18,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:18,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:18,196 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-27 03:54:18,196 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@56199911[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:18,196 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-27 03:54:18,196 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,198 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,198 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-27 03:54:18,198 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-27 03:54:18,198 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:18,198 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,198 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:18,201 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:18,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:18,201 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-27 03:54:18,201 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:18,201 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-27 03:54:18,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:18,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:18,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:18,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:18,204 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:54:18,204 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-27 03:54:18,204 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:18,204 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:18,204 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,204 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:18,206 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,206 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,206 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:18,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:18,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:18,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:18,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:18,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:18,207 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:18,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,209 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:18,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,211 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,211 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:18,211 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:18,211 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,211 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,211 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,211 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,211 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:18,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:18,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:18,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:18,211 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,214 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:18,211 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-27 03:54:18,214 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-27 03:54:18,215 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:18,215 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:18,216 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:18,316 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:18,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:18,324 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:18,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:18,325 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,326 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-27 03:54:18,326 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:18,326 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-27 03:54:18,326 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-27 03:54:18,326 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,326 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,327 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:18,327 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:18,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:18,328 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,328 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:18,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,328 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:18,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,328 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,329 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-27 03:54:18,329 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,329 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-27 03:54:18,329 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:18,329 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-27 03:54:18,329 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:18,329 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:18,329 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:18,329 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. started... 2023-03-27 03:54:18,329 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a7921456431e437526c5b87521696821 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:18,351 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101f718da29ea6644d1bea8a4b69c8380dd to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101f718da29ea6644d1bea8a4b69c8380dd 2023-03-27 03:54:18,362 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/0415aee491674c3f89ecdbf63e6895d3 2023-03-27 03:54:18,368 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/0415aee491674c3f89ecdbf63e6895d3 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/0415aee491674c3f89ecdbf63e6895d3 2023-03-27 03:54:18,376 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/0415aee491674c3f89ecdbf63e6895d3, entries=1, sequenceid=29, filesize=4.9 K 2023-03-27 03:54:18,380 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a7921456431e437526c5b87521696821 in 51ms, sequenceid=29, compaction requested=true 2023-03-27 03:54:18,380 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a7921456431e437526c5b87521696821: 2023-03-27 03:54:18,380 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:18,380 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:18,380 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:18,380 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,380 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-27 03:54:18,380 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-27 03:54:18,383 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,383 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:18,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:18,384 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,384 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:18,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:18,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:18,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:18,386 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-27 03:54:18,386 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5782205f[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:18,386 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-27 03:54:18,386 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,388 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,388 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-27 03:54:18,388 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-27 03:54:18,388 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:18,388 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,388 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:18,391 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:18,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:18,391 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-27 03:54:18,392 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:18,392 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-27 03:54:18,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:18,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:18,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:18,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:18,398 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:54:18,398 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-27 03:54:18,398 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:18,398 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:18,398 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,398 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:18,401 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,401 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:18,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,401 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:18,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:18,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:18,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:18,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:18,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,404 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:18,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,406 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:18,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:18,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:18,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:18,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:18,410 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:18,410 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-27 03:54:18,411 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:18,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,411 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,411 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:18,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:18,411 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:18,411 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:18,511 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:18,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:18,517 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:18,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:18,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,518 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-27 03:54:18,519 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:18,519 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-27 03:54:18,519 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-27 03:54:18,519 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,519 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,523 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:18,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:18,523 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:18,523 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,523 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:18,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,524 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:18,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,524 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,524 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-27 03:54:18,525 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,525 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-27 03:54:18,525 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:18,525 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-27 03:54:18,525 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:18,525 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:18,525 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:18,525 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. started... 2023-03-27 03:54:18,525 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a7921456431e437526c5b87521696821 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:18,554 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010325082f43b94649108c3d5f243e35898d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010325082f43b94649108c3d5f243e35898d 2023-03-27 03:54:18,573 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-27 03:54:18,978 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/93a2ff22fa2143fcaa553b7b32dd9249 2023-03-27 03:54:18,984 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/93a2ff22fa2143fcaa553b7b32dd9249 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/93a2ff22fa2143fcaa553b7b32dd9249 2023-03-27 03:54:18,989 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/93a2ff22fa2143fcaa553b7b32dd9249, entries=1, sequenceid=33, filesize=4.9 K 2023-03-27 03:54:18,990 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a7921456431e437526c5b87521696821 in 465ms, sequenceid=33, compaction requested=true 2023-03-27 03:54:18,990 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a7921456431e437526c5b87521696821: 2023-03-27 03:54:18,990 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:18,990 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:18,990 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:18,990 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,990 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-27 03:54:18,990 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-27 03:54:18,992 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:18,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:18,992 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,992 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,992 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:18,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:18,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:18,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:18,994 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-27 03:54:18,995 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@48417ff8[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:18,995 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-27 03:54:18,995 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,997 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,997 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-27 03:54:18,997 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-27 03:54:18,997 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:18,997 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,997 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:18,998 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-27 03:54:18,998 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:18,998 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,998 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-27 03:54:18,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:18,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:18,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:18,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:18,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:18,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:18,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:19,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:19,001 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:54:19,001 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-27 03:54:19,001 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:19,001 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:19,001 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,001 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:19,002 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:19,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:19,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:19,003 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,003 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:19,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,003 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:19,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:19,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:19,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,005 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:19,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,008 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,008 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,008 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:19,008 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:19,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:19,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:19,008 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:19,008 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,008 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-27 03:54:19,008 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:19,008 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,008 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,008 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,008 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,008 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:19,008 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:19,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:19,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:19,109 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:19,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:19,115 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:19,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:19,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,117 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-27 03:54:19,117 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:19,117 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-27 03:54:19,117 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-27 03:54:19,117 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,117 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,120 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:19,120 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:19,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:19,121 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,121 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:19,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,121 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:19,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,122 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,122 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-27 03:54:19,122 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,122 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-27 03:54:19,122 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:19,122 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-27 03:54:19,123 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:19,123 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:19,123 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:19,123 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. started... 2023-03-27 03:54:19,123 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a7921456431e437526c5b87521696821 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:19,150 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090723fddc20022043e1b96d6547b1d946bf to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015090723fddc20022043e1b96d6547b1d946bf 2023-03-27 03:54:19,181 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/869cddc3458a44acbe31282131f69633 2023-03-27 03:54:19,195 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/869cddc3458a44acbe31282131f69633 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/869cddc3458a44acbe31282131f69633 2023-03-27 03:54:19,204 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/869cddc3458a44acbe31282131f69633, entries=1, sequenceid=37, filesize=4.9 K 2023-03-27 03:54:19,206 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a7921456431e437526c5b87521696821 in 83ms, sequenceid=37, compaction requested=true 2023-03-27 03:54:19,206 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a7921456431e437526c5b87521696821: 2023-03-27 03:54:19,206 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:19,206 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:19,206 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:19,206 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,206 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-27 03:54:19,206 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-27 03:54:19,208 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,208 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:19,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:19,209 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,209 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:19,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:19,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:19,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:19,211 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-27 03:54:19,211 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a4e78a3[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:19,211 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-27 03:54:19,211 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,212 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,212 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-27 03:54:19,212 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-27 03:54:19,212 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:19,213 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,213 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:19,214 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-27 03:54:19,214 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,214 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:19,214 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-27 03:54:19,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:19,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:19,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:19,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:19,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:19,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:19,217 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:54:19,217 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-27 03:54:19,217 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:19,217 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:19,217 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,217 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:19,220 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,220 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:19,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,220 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:19,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:19,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:19,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:19,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:19,221 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:19,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,224 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:19,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,227 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,227 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,227 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,227 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:19,227 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:19,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:19,227 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,228 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-27 03:54:19,228 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:19,228 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,228 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,228 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,228 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:19,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:19,228 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:19,228 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:19,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-27 03:54:19,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:19,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:19,328 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:19,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:19,333 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:19,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:19,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,335 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-03-27 03:54:19,335 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:19,335 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-03-27 03:54:19,335 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-03-27 03:54:19,335 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,335 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,337 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:19,337 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:19,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:19,337 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,337 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:19,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,338 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:19,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,338 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,338 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-03-27 03:54:19,338 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,338 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-03-27 03:54:19,338 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:19,338 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-03-27 03:54:19,338 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:19,339 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:19,339 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:19,339 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. started... 2023-03-27 03:54:19,339 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a7921456431e437526c5b87521696821 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:19,352 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120a932a4d369b342b5ade6b07c70a66670 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151120a932a4d369b342b5ade6b07c70a66670 2023-03-27 03:54:19,759 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/8da1a159900b48d6b018ae74c4110ce1 2023-03-27 03:54:19,765 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/8da1a159900b48d6b018ae74c4110ce1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/8da1a159900b48d6b018ae74c4110ce1 2023-03-27 03:54:19,770 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/8da1a159900b48d6b018ae74c4110ce1, entries=1, sequenceid=41, filesize=4.9 K 2023-03-27 03:54:19,771 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a7921456431e437526c5b87521696821 in 432ms, sequenceid=41, compaction requested=true 2023-03-27 03:54:19,771 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a7921456431e437526c5b87521696821: 2023-03-27 03:54:19,772 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:19,772 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:19,772 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:19,772 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,772 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-03-27 03:54:19,772 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-03-27 03:54:19,774 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,774 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:19,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:19,774 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,774 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:19,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:19,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:19,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:19,776 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-03-27 03:54:19,776 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-03-27 03:54:19,776 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5abea[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:19,776 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,779 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,779 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-03-27 03:54:19,779 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-03-27 03:54:19,779 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:19,779 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,780 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:19,781 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:19,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:19,781 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-03-27 03:54:19,781 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:19,781 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-03-27 03:54:19,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:19,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:19,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:19,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:19,784 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:54:19,784 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-03-27 03:54:19,784 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:19,784 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:19,784 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,784 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:19,786 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,786 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:19,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:19,786 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:19,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:19,786 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:19,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:19,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:19,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,788 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:19,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,794 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,794 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:19,794 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:19,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:19,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:19,794 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:19,794 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,794 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-03-27 03:54:19,795 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-03-27 03:54:19,795 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:19,795 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:19,795 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:19,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:19,795 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,795 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:19,795 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,795 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,795 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:19,896 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:19,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:19,898 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38271] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-03-27 03:54:19,906 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-27 03:54:19,907 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-27 03:54:19,910 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:54:19,910 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679889859910 2023-03-27 03:54:19,912 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-27 03:54:19,913 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-03-27 03:54:19,913 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-27 03:54:19,914 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] zookeeper.ReadOnlyZKClient(139): Connect 0x343d14c4 to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:54:19,932 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@348309e3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:54:19,932 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-27 03:54:20,454 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ccf1e9725fd94f25a06094aa83e3c27c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103ccf1e9725fd94f25a06094aa83e3c27c 2023-03-27 03:54:20,455 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-27 03:54:20,474 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-27 03:54:20,476 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36074, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-27 03:54:20,489 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-27 03:54:20,490 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49628, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-27 03:54:20,499 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/e391034e47e84c718a6693d4435c309e first=Optional[r3] last=Optional[r6] 2023-03-27 03:54:20,500 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/e391034e47e84c718a6693d4435c309e}] 2023-03-27 03:54:20,539 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-27 03:54:20,539 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1ced8ffe to 127.0.0.1:62246 2023-03-27 03:54:20,539 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:54:20,540 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-27 03:54:20,540 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2f4dd525 to 127.0.0.1:62246 2023-03-27 03:54:20,540 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:54:20,540 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-27 03:54:20,540 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4ff36841 to 127.0.0.1:62246 2023-03-27 03:54:20,540 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:54:20,541 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/e391034e47e84c718a6693d4435c309e for inclusion in a7921456431e437526c5b87521696821/family1 2023-03-27 03:54:20,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-27 03:54:20,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last= 2023-03-27 03:54:20,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for a7921456431e437526c5b87521696821: 2023-03-27 03:54:20,546 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/e391034e47e84c718a6693d4435c309e to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyPolicy__etja1l7uckfco6lrasr1jp5p1et3vfcin75etd9b14em29smurtdb5q7gk1oep1l/family1/e391034e47e84c718a6693d4435c309e 2023-03-27 03:54:20,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyPolicy__etja1l7uckfco6lrasr1jp5p1et3vfcin75etd9b14em29smurtdb5q7gk1oep1l/family1/e391034e47e84c718a6693d4435c309e as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/514b9431f3a6405190c82e1c8958bb00_SeqId_44_ 2023-03-27 03:54:20,549 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyPolicy__etja1l7uckfco6lrasr1jp5p1et3vfcin75etd9b14em29smurtdb5q7gk1oep1l/family1/e391034e47e84c718a6693d4435c309e into a7921456431e437526c5b87521696821/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/514b9431f3a6405190c82e1c8958bb00_SeqId_44_ - updating store file list. 2023-03-27 03:54:20,553 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/514b9431f3a6405190c82e1c8958bb00_SeqId_44_ into a7921456431e437526c5b87521696821/family1 2023-03-27 03:54:20,553 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyPolicy__etja1l7uckfco6lrasr1jp5p1et3vfcin75etd9b14em29smurtdb5q7gk1oep1l/family1/e391034e47e84c718a6693d4435c309e into a7921456431e437526c5b87521696821/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/514b9431f3a6405190c82e1c8958bb00_SeqId_44_) 2023-03-27 03:54:20,553 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyPolicy__etja1l7uckfco6lrasr1jp5p1et3vfcin75etd9b14em29smurtdb5q7gk1oep1l/family1/e391034e47e84c718a6693d4435c309e 2023-03-27 03:54:20,553 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-27 03:54:20,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-27 03:54:20,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(1912): a7921456431e437526c5b87521696821/family1 is initiating minor compaction 2023-03-27 03:54:20,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4548af2b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:20,558 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a7921456431e437526c5b87521696821/family1 in testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:20,558 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/884f448a7ab54f44848b58c9b3fe3890, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/c4fa0e28d8f34a1b87cca9de0f92bb40, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/c8f53d60329a475fac430613588f427c, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/73c1a4cb91684d718f1d42c87264d0d8, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/f61d186cbc5d4c2cad8df022e7a6dd35, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/0415aee491674c3f89ecdbf63e6895d3, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/93a2ff22fa2143fcaa553b7b32dd9249, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/869cddc3458a44acbe31282131f69633, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/8da1a159900b48d6b018ae74c4110ce1, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/514b9431f3a6405190c82e1c8958bb00_SeqId_44_] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp, totalSize=50.0 K 2023-03-27 03:54:20,558 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 884f448a7ab54f44848b58c9b3fe3890, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-27 03:54:20,558 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting c4fa0e28d8f34a1b87cca9de0f92bb40, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-27 03:54:20,558 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting c8f53d60329a475fac430613588f427c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-27 03:54:20,558 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 73c1a4cb91684d718f1d42c87264d0d8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-27 03:54:20,558 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting f61d186cbc5d4c2cad8df022e7a6dd35, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-27 03:54:20,559 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 0415aee491674c3f89ecdbf63e6895d3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-27 03:54:20,559 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 93a2ff22fa2143fcaa553b7b32dd9249, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-27 03:54:20,559 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 869cddc3458a44acbe31282131f69633, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-27 03:54:20,559 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 8da1a159900b48d6b018ae74c4110ce1, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-27 03:54:20,559 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 514b9431f3a6405190c82e1c8958bb00_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-27 03:54:20,561 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-27 03:54:20,562 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:20,563 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228d319b6ebb33a4444b3db74f0c1749a2c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228d319b6ebb33a4444b3db74f0c1749a2c 2023-03-27 03:54:20,565 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231991479bb45e347d0a6d84a7203e16c1c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231991479bb45e347d0a6d84a7203e16c1c 2023-03-27 03:54:20,566 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101f718da29ea6644d1bea8a4b69c8380dd to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101f718da29ea6644d1bea8a4b69c8380dd 2023-03-27 03:54:20,568 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010325082f43b94649108c3d5f243e35898d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2016010325082f43b94649108c3d5f243e35898d 2023-03-27 03:54:20,568 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-03-27 03:54:20,568 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-27 03:54:20,568 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-27 03:54:20,568 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-27 03:54:20,569 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-27 03:54:20,569 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-27 03:54:20,569 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-27 03:54:20,569 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-27 03:54:20,569 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-27 03:54:20,569 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-27 03:54:20,607 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] throttle.PressureAwareThroughputController(145): a7921456431e437526c5b87521696821#family1#compaction#49 average throughput is 0.23 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:54:21,053 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/ad282c8d39d141fb98d23e7298097896 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/ad282c8d39d141fb98d23e7298097896 2023-03-27 03:54:21,055 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512052f4abb6518f34ea8b97dd109f7f185db to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512052f4abb6518f34ea8b97dd109f7f185db 2023-03-27 03:54:21,056 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-27 03:54:21,059 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in a7921456431e437526c5b87521696821/family1 of a7921456431e437526c5b87521696821 into ad282c8d39d141fb98d23e7298097896(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:54:21,059 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a7921456431e437526c5b87521696821: 2023-03-27 03:54:21,060 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1679889260558; duration=0sec 2023-03-27 03:54:21,060 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:21,089 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f86abf09041447af828f93dcda5e9ca6 first=Optional[r00] last=Optional[r2] 2023-03-27 03:54:21,090 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f86abf09041447af828f93dcda5e9ca6}] 2023-03-27 03:54:21,092 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f86abf09041447af828f93dcda5e9ca6 for inclusion in a7921456431e437526c5b87521696821/family1 2023-03-27 03:54:21,102 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/884f448a7ab54f44848b58c9b3fe3890, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/c4fa0e28d8f34a1b87cca9de0f92bb40, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/c8f53d60329a475fac430613588f427c, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/73c1a4cb91684d718f1d42c87264d0d8, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/f61d186cbc5d4c2cad8df022e7a6dd35, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/0415aee491674c3f89ecdbf63e6895d3, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/93a2ff22fa2143fcaa553b7b32dd9249, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/869cddc3458a44acbe31282131f69633, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/8da1a159900b48d6b018ae74c4110ce1, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/514b9431f3a6405190c82e1c8958bb00_SeqId_44_] to archive 2023-03-27 03:54:21,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-27 03:54:21,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last= 2023-03-27 03:54:21,104 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:21,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for a7921456431e437526c5b87521696821: 2023-03-27 03:54:21,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/f86abf09041447af828f93dcda5e9ca6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyPolicy__qmoj5vl9g9ik97ns8o5af88p060lhn4u3rtbdv4ohsd49lclk4pj263gartnlh4p/family1/f86abf09041447af828f93dcda5e9ca6 2023-03-27 03:54:21,106 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/884f448a7ab54f44848b58c9b3fe3890 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/884f448a7ab54f44848b58c9b3fe3890 2023-03-27 03:54:21,109 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/c4fa0e28d8f34a1b87cca9de0f92bb40 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/c4fa0e28d8f34a1b87cca9de0f92bb40 2023-03-27 03:54:21,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyPolicy__qmoj5vl9g9ik97ns8o5af88p060lhn4u3rtbdv4ohsd49lclk4pj263gartnlh4p/family1/f86abf09041447af828f93dcda5e9ca6 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/ae69cfdb8eb24720893945013f73e41a_SeqId_47_ 2023-03-27 03:54:21,111 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyPolicy__qmoj5vl9g9ik97ns8o5af88p060lhn4u3rtbdv4ohsd49lclk4pj263gartnlh4p/family1/f86abf09041447af828f93dcda5e9ca6 into a7921456431e437526c5b87521696821/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/ae69cfdb8eb24720893945013f73e41a_SeqId_47_ - updating store file list. 2023-03-27 03:54:21,111 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/c8f53d60329a475fac430613588f427c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/c8f53d60329a475fac430613588f427c 2023-03-27 03:54:21,113 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/73c1a4cb91684d718f1d42c87264d0d8 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/73c1a4cb91684d718f1d42c87264d0d8 2023-03-27 03:54:21,115 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/f61d186cbc5d4c2cad8df022e7a6dd35 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/f61d186cbc5d4c2cad8df022e7a6dd35 2023-03-27 03:54:21,116 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/ae69cfdb8eb24720893945013f73e41a_SeqId_47_ into a7921456431e437526c5b87521696821/family1 2023-03-27 03:54:21,116 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyPolicy__qmoj5vl9g9ik97ns8o5af88p060lhn4u3rtbdv4ohsd49lclk4pj263gartnlh4p/family1/f86abf09041447af828f93dcda5e9ca6 into a7921456431e437526c5b87521696821/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/ae69cfdb8eb24720893945013f73e41a_SeqId_47_) 2023-03-27 03:54:21,117 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/0415aee491674c3f89ecdbf63e6895d3 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/0415aee491674c3f89ecdbf63e6895d3 2023-03-27 03:54:21,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyPolicy__qmoj5vl9g9ik97ns8o5af88p060lhn4u3rtbdv4ohsd49lclk4pj263gartnlh4p/family1/f86abf09041447af828f93dcda5e9ca6 2023-03-27 03:54:21,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-27 03:54:21,119 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/93a2ff22fa2143fcaa553b7b32dd9249 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/93a2ff22fa2143fcaa553b7b32dd9249 2023-03-27 03:54:21,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 345120 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-27 03:54:21,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(1912): a7921456431e437526c5b87521696821/family1 is initiating minor compaction (all files) 2023-03-27 03:54:21,120 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a7921456431e437526c5b87521696821/family1 in testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:54:21,120 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/071a858929244c46a803bdda5f31810d, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/ad282c8d39d141fb98d23e7298097896, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/ae69cfdb8eb24720893945013f73e41a_SeqId_47_] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp, totalSize=337.0 K 2023-03-27 03:54:21,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1a22bac6; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:21,122 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 071a858929244c46a803bdda5f31810d, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878842162 2023-03-27 03:54:21,123 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting ad282c8d39d141fb98d23e7298097896, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621242162 2023-03-27 03:54:21,123 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/869cddc3458a44acbe31282131f69633 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/869cddc3458a44acbe31282131f69633 2023-03-27 03:54:21,123 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting ae69cfdb8eb24720893945013f73e41a_SeqId_47_, keycount=1002, bloomtype=ROW, size=165.4 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878842162 2023-03-27 03:54:21,124 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-27 03:54:21,126 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:21,128 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/8da1a159900b48d6b018ae74c4110ce1 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/8da1a159900b48d6b018ae74c4110ce1 2023-03-27 03:54:21,132 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/514b9431f3a6405190c82e1c8958bb00_SeqId_44_ to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/514b9431f3a6405190c82e1c8958bb00_SeqId_44_ 2023-03-27 03:54:21,136 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130cc7f3c46eeb045f1890ca8f502d446c0 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130cc7f3c46eeb045f1890ca8f502d446c0 2023-03-27 03:54:21,138 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512010afead3c5846488b87b93f3473c8aba0 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512010afead3c5846488b87b93f3473c8aba0 2023-03-27 03:54:21,139 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205d63c758208174e5b94b25ef8efb746d6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205d63c758208174e5b94b25ef8efb746d6 2023-03-27 03:54:21,139 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-03-27 03:54:21,139 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-27 03:54:21,140 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-03-27 03:54:21,140 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-03-27 03:54:21,140 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:54:21,153 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-03-27 03:54:21,158 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.2530 sec 2023-03-27 03:54:21,167 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] throttle.PressureAwareThroughputController(145): a7921456431e437526c5b87521696821#family1#compaction#50 average throughput is 4.99 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:54:21,279 INFO [Listener at localhost/36131] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=266 (was 253) - Thread LEAK? -, OpenFileDescriptor=637 (was 597) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=470 (was 450) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=8806 (was 8875) 2023-03-27 03:54:21,293 INFO [Listener at localhost/36131] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=266, OpenFileDescriptor=637, MaxFileDescriptor=60000, SystemLoadAverage=470, ProcessCount=170, AvailableMemoryMB=8805 2023-03-27 03:54:21,296 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-27 03:54:21,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:21,298 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-03-27 03:54:21,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-03-27 03:54:21,299 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-27 03:54:21,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-27 03:54:21,300 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:21,301 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd empty. 2023-03-27 03:54:21,302 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:21,302 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-03-27 03:54:21,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-27 03:54:21,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-27 03:54:21,711 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-27 03:54:21,714 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7675): creating {ENCODED => a03c85498f16edb6952ae7ec4494cfbd, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp 2023-03-27 03:54:21,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-27 03:54:22,013 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/.tmp/family1/699ce8a270f14a2e9ea4cef4a310d5d8 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/699ce8a270f14a2e9ea4cef4a310d5d8 2023-03-27 03:54:22,022 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in a7921456431e437526c5b87521696821/family1 of a7921456431e437526c5b87521696821 into 699ce8a270f14a2e9ea4cef4a310d5d8(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:54:22,022 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a7921456431e437526c5b87521696821: 2023-03-27 03:54:22,022 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821., storeName=family1, fileCount=3, fileSize=337.0 K (165.1 K, 6.6 K, 165.4 K), priority=2, time=1679889261119; duration=0sec 2023-03-27 03:54:22,022 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:22,093 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/071a858929244c46a803bdda5f31810d, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/ad282c8d39d141fb98d23e7298097896, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/ae69cfdb8eb24720893945013f73e41a_SeqId_47_] to archive 2023-03-27 03:54:22,094 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:22,096 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/071a858929244c46a803bdda5f31810d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/071a858929244c46a803bdda5f31810d 2023-03-27 03:54:22,098 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/ad282c8d39d141fb98d23e7298097896 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/ad282c8d39d141fb98d23e7298097896 2023-03-27 03:54:22,099 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/ae69cfdb8eb24720893945013f73e41a_SeqId_47_ to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/family1/ae69cfdb8eb24720893945013f73e41a_SeqId_47_ 2023-03-27 03:54:22,134 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:22,134 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1604): Closing a03c85498f16edb6952ae7ec4494cfbd, disabling compactions & flushes 2023-03-27 03:54:22,134 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:22,134 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:22,134 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. after waiting 0 ms 2023-03-27 03:54:22,134 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:22,135 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:22,135 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1558): Region close journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:54:22,137 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-03-27 03:54:22,138 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679889262137"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889262137"}]},"ts":"1679889262137"} 2023-03-27 03:54:22,139 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-27 03:54:22,140 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-27 03:54:22,140 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889262140"}]},"ts":"1679889262140"} 2023-03-27 03:54:22,142 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-03-27 03:54:22,145 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a03c85498f16edb6952ae7ec4494cfbd, ASSIGN}] 2023-03-27 03:54:22,147 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a03c85498f16edb6952ae7ec4494cfbd, ASSIGN 2023-03-27 03:54:22,148 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a03c85498f16edb6952ae7ec4494cfbd, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=false 2023-03-27 03:54:22,299 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=a03c85498f16edb6952ae7ec4494cfbd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:22,300 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679889262299"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889262299"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889262299"}]},"ts":"1679889262299"} 2023-03-27 03:54:22,302 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure a03c85498f16edb6952ae7ec4494cfbd, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:54:22,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-27 03:54:22,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:22,457 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a03c85498f16edb6952ae7ec4494cfbd, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.', STARTKEY => '', ENDKEY => ''} 2023-03-27 03:54:22,457 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:22,458 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:22,458 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:22,458 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:22,461 INFO [StoreOpener-a03c85498f16edb6952ae7ec4494cfbd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:22,464 DEBUG [StoreOpener-a03c85498f16edb6952ae7ec4494cfbd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1 2023-03-27 03:54:22,464 DEBUG [StoreOpener-a03c85498f16edb6952ae7ec4494cfbd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1 2023-03-27 03:54:22,464 INFO [StoreOpener-a03c85498f16edb6952ae7ec4494cfbd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a03c85498f16edb6952ae7ec4494cfbd columnFamilyName family1 2023-03-27 03:54:22,466 INFO [StoreOpener-a03c85498f16edb6952ae7ec4494cfbd-1] regionserver.HStore(310): Store=a03c85498f16edb6952ae7ec4494cfbd/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:54:22,467 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:22,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:22,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:22,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:54:22,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a03c85498f16edb6952ae7ec4494cfbd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59508809, jitterRate=-0.11324964463710785}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-27 03:54:22,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:54:22,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd., pid=41, masterSystemTime=1679889262454 2023-03-27 03:54:22,479 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=a03c85498f16edb6952ae7ec4494cfbd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:22,479 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679889262479"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889262479"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889262479"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889262479"}]},"ts":"1679889262479"} 2023-03-27 03:54:22,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:22,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:22,483 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-03-27 03:54:22,483 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure a03c85498f16edb6952ae7ec4494cfbd, server=jenkins-hbase4.apache.org,39727,1679889218560 in 180 msec 2023-03-27 03:54:22,489 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-03-27 03:54:22,490 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a03c85498f16edb6952ae7ec4494cfbd, ASSIGN in 339 msec 2023-03-27 03:54:22,490 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-27 03:54:22,490 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889262490"}]},"ts":"1679889262490"} 2023-03-27 03:54:22,492 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-03-27 03:54:22,494 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-03-27 03:54:22,495 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1980 sec 2023-03-27 03:54:22,866 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-27 03:54:23,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-03-27 03:54:23,405 INFO [Listener at localhost/36131] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-03-27 03:54:23,427 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:23,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:23,452 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-27 03:54:23,452 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:23,452 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-27 03:54:23,452 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-27 03:54:23,453 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:23,453 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:23,454 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:23,454 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:23,455 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:23,455 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:23,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:23,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:23,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:23,456 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:23,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:23,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:23,457 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:23,459 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:23,459 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-27 03:54:23,459 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:23,459 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-27 03:54:23,464 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:23,464 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:23,464 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:23,464 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. started... 2023-03-27 03:54:23,465 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a03c85498f16edb6952ae7ec4494cfbd 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-27 03:54:23,970 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113049acd9f8e9964b32a5d062ec09316966 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015113049acd9f8e9964b32a5d062ec09316966 2023-03-27 03:54:24,374 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/29fe9f0c28914c6d8f434d9e1c8da664 2023-03-27 03:54:24,380 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/29fe9f0c28914c6d8f434d9e1c8da664 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/29fe9f0c28914c6d8f434d9e1c8da664 2023-03-27 03:54:24,384 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/29fe9f0c28914c6d8f434d9e1c8da664, entries=1000, sequenceid=5, filesize=191.4 K 2023-03-27 03:54:24,385 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for a03c85498f16edb6952ae7ec4494cfbd in 920ms, sequenceid=5, compaction requested=false 2023-03-27 03:54:24,386 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:54:24,386 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:24,386 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:24,386 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:24,386 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:24,386 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-27 03:54:24,386 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-27 03:54:24,389 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:24,389 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:24,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:24,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:24,389 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,389 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:24,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:24,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:24,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:24,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:24,391 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-27 03:54:24,391 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@46c3f36d[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:24,391 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-27 03:54:24,391 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,393 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,393 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-27 03:54:24,393 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-27 03:54:24,393 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:24,395 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-27 03:54:24,395 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:24,395 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-27 03:54:24,395 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:24,395 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:24,395 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:24,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:24,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:24,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:24,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:24,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:24,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:24,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:24,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:24,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:24,403 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-27 03:54:24,403 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-27 03:54:24,403 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:24,403 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:24,403 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,403 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:24,405 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,405 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:24,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,405 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:24,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:24,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:24,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:24,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:24,406 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:24,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:24,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,407 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:24,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:24,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:24,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:24,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:24,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:24,410 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-27 03:54:24,410 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:24,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:24,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,410 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-27 03:54:24,410 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:24,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:24,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:24,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,411 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,411 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:24,411 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:24,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:24,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:24,511 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:24,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:24,516 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:24,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:24,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,518 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-27 03:54:24,518 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:24,518 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-27 03:54:24,518 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-27 03:54:24,519 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,519 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,521 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:24,521 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:24,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:24,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:24,521 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:24,521 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:24,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,522 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:24,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,522 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,523 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-27 03:54:24,523 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:24,523 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-27 03:54:24,523 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:24,523 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-27 03:54:24,524 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:24,524 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:24,524 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:24,524 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. started... 2023-03-27 03:54:24,524 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a03c85498f16edb6952ae7ec4494cfbd 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-27 03:54:24,938 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511289db2ad7cca3249d188f6cdfddbb920ee to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511289db2ad7cca3249d188f6cdfddbb920ee 2023-03-27 03:54:25,346 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/b758b804dfab49a4983571f13ebb4b83 2023-03-27 03:54:25,352 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/b758b804dfab49a4983571f13ebb4b83 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/b758b804dfab49a4983571f13ebb4b83 2023-03-27 03:54:25,357 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/b758b804dfab49a4983571f13ebb4b83, entries=1, sequenceid=9, filesize=5.0 K 2023-03-27 03:54:25,358 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for a03c85498f16edb6952ae7ec4494cfbd in 834ms, sequenceid=9, compaction requested=false 2023-03-27 03:54:25,358 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:54:25,358 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:25,358 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:25,358 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:25,358 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:25,358 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-27 03:54:25,358 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-27 03:54:25,359 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:25,360 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:25,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:25,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:25,360 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,360 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:25,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:25,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:25,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:25,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:25,361 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-27 03:54:25,361 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53efaf2[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:25,363 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-27 03:54:25,363 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,364 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,364 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-27 03:54:25,364 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-27 03:54:25,364 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:25,364 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:25,364 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:25,367 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:25,367 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-27 03:54:25,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:25,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:25,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:25,367 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:25,367 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-27 03:54:25,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:25,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:25,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:25,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:25,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:25,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:25,370 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-27 03:54:25,370 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-27 03:54:25,370 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:25,370 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:25,370 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,370 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:25,372 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,372 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:25,372 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:25,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:25,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:25,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:25,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:25,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,373 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:25,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:25,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,375 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:25,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:25,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:25,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:25,385 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:25,385 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:25,385 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:25,386 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:25,386 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,386 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:25,386 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:25,386 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-27 03:54:25,386 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:25,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:25,387 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-27 03:54:25,386 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,387 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,387 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:25,387 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,387 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:25,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:25,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:25,487 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:25,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:25,493 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:25,494 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:25,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,495 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-27 03:54:25,495 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:25,495 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-27 03:54:25,495 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-27 03:54:25,495 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,495 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,497 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:25,497 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:25,497 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:25,497 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:25,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:25,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:25,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,499 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:25,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,500 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,500 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-27 03:54:25,500 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:25,500 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-27 03:54:25,500 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:25,500 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-27 03:54:25,500 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:25,500 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:25,500 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:25,500 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. started... 2023-03-27 03:54:25,501 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a03c85498f16edb6952ae7ec4494cfbd 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:25,924 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201dc32484c31d24531865b3a8cc0eeb37a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201dc32484c31d24531865b3a8cc0eeb37a 2023-03-27 03:54:26,330 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/6062d443d71b49c590dfcde40f365b02 2023-03-27 03:54:26,336 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/6062d443d71b49c590dfcde40f365b02 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/6062d443d71b49c590dfcde40f365b02 2023-03-27 03:54:26,340 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/6062d443d71b49c590dfcde40f365b02, entries=1, sequenceid=13, filesize=5.0 K 2023-03-27 03:54:26,341 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a03c85498f16edb6952ae7ec4494cfbd in 840ms, sequenceid=13, compaction requested=true 2023-03-27 03:54:26,341 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:54:26,341 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:26,342 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:26,342 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:26,342 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:26,342 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-27 03:54:26,343 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-27 03:54:26,344 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:26,344 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:26,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:26,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:26,346 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,346 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:26,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:26,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:26,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:26,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:26,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-27 03:54:26,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c5c1196[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:26,347 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-27 03:54:26,347 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,348 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,348 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-27 03:54:26,348 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-27 03:54:26,348 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:26,348 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:26,348 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:26,350 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-27 03:54:26,350 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:26,350 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:26,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:26,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:26,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:26,351 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-27 03:54:26,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:26,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:26,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:26,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:26,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:26,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:26,353 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-27 03:54:26,353 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-27 03:54:26,353 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:26,353 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:26,353 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,353 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:26,354 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,355 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:26,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,355 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:26,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:26,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:26,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:26,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:26,355 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:26,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:26,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,356 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:26,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:26,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:26,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:26,358 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:26,358 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:26,358 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,358 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:26,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:26,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:26,359 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:26,359 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,359 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-27 03:54:26,359 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:26,359 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-27 03:54:26,359 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:26,359 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,359 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,359 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:26,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:26,359 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:26,359 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:26,459 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:26,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:26,464 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:26,466 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:26,466 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,466 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-27 03:54:26,466 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:26,466 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-27 03:54:26,466 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-27 03:54:26,467 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,467 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,468 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:26,468 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:26,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:26,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:26,468 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:26,468 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:26,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,469 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:26,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,469 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,469 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-27 03:54:26,469 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:26,469 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-27 03:54:26,469 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:26,470 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-27 03:54:26,470 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:26,470 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:26,470 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:26,470 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. started... 2023-03-27 03:54:26,470 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a03c85498f16edb6952ae7ec4494cfbd 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:26,888 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205ac6762dc8d7a4d7cada9be949325d91f to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205ac6762dc8d7a4d7cada9be949325d91f 2023-03-27 03:54:27,295 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/a2187447aca34853aaca274aa10e0fdb 2023-03-27 03:54:27,300 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/a2187447aca34853aaca274aa10e0fdb as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/a2187447aca34853aaca274aa10e0fdb 2023-03-27 03:54:27,305 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/a2187447aca34853aaca274aa10e0fdb, entries=1, sequenceid=17, filesize=5.0 K 2023-03-27 03:54:27,306 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a03c85498f16edb6952ae7ec4494cfbd in 836ms, sequenceid=17, compaction requested=true 2023-03-27 03:54:27,306 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:54:27,306 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:27,306 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:27,306 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:27,306 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,306 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-27 03:54:27,307 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-27 03:54:27,308 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,308 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:27,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:27,309 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,309 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:27,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:27,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:27,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:27,310 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-27 03:54:27,311 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@412bad0c[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:27,311 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-27 03:54:27,311 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,312 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,312 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-27 03:54:27,312 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-27 03:54:27,312 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:27,312 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,312 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:27,315 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,315 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-27 03:54:27,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:27,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:27,315 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:27,315 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-27 03:54:27,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:27,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:27,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:27,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:27,318 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-27 03:54:27,318 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-27 03:54:27,318 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:27,318 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:27,318 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,318 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:27,323 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,323 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:27,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:27,323 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:27,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,323 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:27,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:27,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:27,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,324 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:27,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,325 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:27,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,327 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:27,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:27,327 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:27,327 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,327 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,327 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:27,331 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,331 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,331 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:27,331 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,331 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,331 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-27 03:54:27,331 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:27,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:27,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:27,332 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:27,332 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:27,432 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:27,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:27,437 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:27,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:27,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,439 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-27 03:54:27,439 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:27,439 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-27 03:54:27,439 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-27 03:54:27,440 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,440 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,441 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:27,441 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:27,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:27,441 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,441 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:27,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,442 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:27,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,442 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,442 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-27 03:54:27,442 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,443 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-27 03:54:27,443 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:27,443 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-27 03:54:27,443 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:27,443 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:27,443 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:27,444 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. started... 2023-03-27 03:54:27,444 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a03c85498f16edb6952ae7ec4494cfbd 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:27,465 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228e559b0197bfe4e5282943b1500fb1ba4 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228e559b0197bfe4e5282943b1500fb1ba4 2023-03-27 03:54:27,874 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/29abc1b1b5f547aaa1084a394eb04bf2 2023-03-27 03:54:27,879 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/29abc1b1b5f547aaa1084a394eb04bf2 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/29abc1b1b5f547aaa1084a394eb04bf2 2023-03-27 03:54:27,884 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/29abc1b1b5f547aaa1084a394eb04bf2, entries=1, sequenceid=21, filesize=5.0 K 2023-03-27 03:54:27,885 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a03c85498f16edb6952ae7ec4494cfbd in 441ms, sequenceid=21, compaction requested=true 2023-03-27 03:54:27,885 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:54:27,885 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:27,885 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:27,885 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:27,885 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,885 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-27 03:54:27,885 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-27 03:54:27,894 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,895 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:27,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:27,895 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,895 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:27,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:27,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:27,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:27,897 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-27 03:54:27,897 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@182a76df[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:27,897 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-27 03:54:27,897 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,898 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,898 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-27 03:54:27,898 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-27 03:54:27,898 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:27,898 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,898 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:27,899 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-27 03:54:27,899 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,899 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:27,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:27,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:27,899 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-27 03:54:27,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:27,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:27,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:27,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:27,902 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-27 03:54:27,902 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-27 03:54:27,902 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:27,902 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:27,902 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,902 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:27,904 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,904 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:27,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:27,904 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:27,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:27,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:27,905 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:27,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:27,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,906 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:27,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,909 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,909 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,909 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,909 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:27,909 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,909 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-27 03:54:27,909 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:27,909 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-27 03:54:27,909 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:27,909 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:27,909 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:27,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:27,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:27,909 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:27,910 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:27,910 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:27,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:27,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:27,959 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-27 03:54:28,010 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:28,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:28,016 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:28,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:28,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,017 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-27 03:54:28,017 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:28,017 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-27 03:54:28,017 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-27 03:54:28,018 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,018 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,019 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:28,019 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:28,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:28,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:28,019 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:28,019 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:28,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,020 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:28,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,020 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,020 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-27 03:54:28,021 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,021 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-27 03:54:28,021 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:28,021 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-27 03:54:28,021 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:28,021 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:28,021 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:28,021 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. started... 2023-03-27 03:54:28,021 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a03c85498f16edb6952ae7ec4494cfbd 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:28,436 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123106d65b6618634ec5a81875564fceeb16 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015123106d65b6618634ec5a81875564fceeb16 2023-03-27 03:54:28,847 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/337dffba0e4443388532170bb94328ac 2023-03-27 03:54:28,853 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/337dffba0e4443388532170bb94328ac as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/337dffba0e4443388532170bb94328ac 2023-03-27 03:54:28,857 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/337dffba0e4443388532170bb94328ac, entries=1, sequenceid=25, filesize=5.0 K 2023-03-27 03:54:28,858 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a03c85498f16edb6952ae7ec4494cfbd in 837ms, sequenceid=25, compaction requested=true 2023-03-27 03:54:28,858 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:54:28,858 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:28,858 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:28,858 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:28,858 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:28,858 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-27 03:54:28,858 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-27 03:54:28,860 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:28,860 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:28,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:28,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:28,860 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:28,860 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:28,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:28,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:28,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:28,861 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-27 03:54:28,861 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a31e642[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:28,861 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-27 03:54:28,862 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,862 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,863 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-27 03:54:28,863 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-27 03:54:28,863 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:28,863 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:28,863 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:28,864 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-27 03:54:28,864 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:28,864 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:28,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:28,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:28,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:28,864 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-27 03:54:28,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:28,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:28,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:28,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:28,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:28,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:28,867 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-27 03:54:28,867 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-27 03:54:28,867 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:28,867 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:28,867 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,867 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:28,870 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,870 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,870 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:28,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:28,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:28,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:28,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:28,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:28,871 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:28,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:28,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,872 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:28,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:28,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:28,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:28,874 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:28,874 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:28,874 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:28,874 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:28,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:28,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:28,874 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-27 03:54:28,874 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-27 03:54:28,875 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:28,874 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,875 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,875 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:28,875 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:28,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:28,875 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,875 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:28,875 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,875 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:28,975 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:28,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:28,981 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:28,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:28,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,983 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-27 03:54:28,983 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:28,983 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-27 03:54:28,983 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-27 03:54:28,984 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,984 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,985 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:28,985 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:28,985 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:28,985 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:28,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:28,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:28,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,986 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:28,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,987 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,987 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-27 03:54:28,987 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:28,987 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-27 03:54:28,987 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:28,987 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-27 03:54:28,987 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:28,987 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:28,987 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:28,987 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. started... 2023-03-27 03:54:28,988 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a03c85498f16edb6952ae7ec4494cfbd 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:29,411 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101a5a638b0ab7c405ebaed30957982aad9 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101a5a638b0ab7c405ebaed30957982aad9 2023-03-27 03:54:29,820 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/a200694ff6c347ad8a0c0162bf758bf9 2023-03-27 03:54:29,825 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/a200694ff6c347ad8a0c0162bf758bf9 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/a200694ff6c347ad8a0c0162bf758bf9 2023-03-27 03:54:29,829 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/a200694ff6c347ad8a0c0162bf758bf9, entries=1, sequenceid=29, filesize=5.0 K 2023-03-27 03:54:29,830 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a03c85498f16edb6952ae7ec4494cfbd in 842ms, sequenceid=29, compaction requested=true 2023-03-27 03:54:29,830 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:54:29,831 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:29,831 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:29,831 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:29,831 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:29,831 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-27 03:54:29,831 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-27 03:54:29,832 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:29,832 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:29,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:29,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:29,832 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,832 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:29,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:29,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:29,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:29,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:29,834 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-27 03:54:29,834 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66ed9a33[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:29,834 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-27 03:54:29,834 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,835 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,835 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-27 03:54:29,835 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-27 03:54:29,835 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:29,835 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:29,835 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:29,836 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:29,836 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-27 03:54:29,837 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:29,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:29,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:29,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:29,837 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-27 03:54:29,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:29,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:29,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:29,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:29,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:29,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:29,839 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-27 03:54:29,839 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-27 03:54:29,839 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:29,839 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:29,839 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,839 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:29,841 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,841 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,841 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:29,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:29,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:29,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:29,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:29,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:29,842 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:29,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:29,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,843 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:29,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:29,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:29,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:29,848 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:29,848 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:29,848 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,848 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:29,848 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-27 03:54:29,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:29,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:29,848 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:29,848 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,848 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-27 03:54:29,849 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:29,848 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:29,849 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,849 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,849 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:29,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:29,849 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:29,849 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:29,949 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:29,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:29,954 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:29,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:29,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,956 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-27 03:54:29,956 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:29,956 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-27 03:54:29,956 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-27 03:54:29,956 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,956 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,958 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:29,958 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:29,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:29,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:29,958 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:29,958 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:29,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,958 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:29,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,959 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,959 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-27 03:54:29,959 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:29,959 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-27 03:54:29,959 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:29,959 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-27 03:54:29,959 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:29,959 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:29,959 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:29,960 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. started... 2023-03-27 03:54:29,960 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a03c85498f16edb6952ae7ec4494cfbd 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:30,374 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601039b00c4bb01b74c9db8613301143bd724 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601039b00c4bb01b74c9db8613301143bd724 2023-03-27 03:54:30,780 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/fb2143f4cab848a590a5998075925b40 2023-03-27 03:54:30,786 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/fb2143f4cab848a590a5998075925b40 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/fb2143f4cab848a590a5998075925b40 2023-03-27 03:54:30,796 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/fb2143f4cab848a590a5998075925b40, entries=1, sequenceid=33, filesize=5.0 K 2023-03-27 03:54:30,800 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a03c85498f16edb6952ae7ec4494cfbd in 840ms, sequenceid=33, compaction requested=true 2023-03-27 03:54:30,800 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:54:30,800 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:30,800 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:30,800 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:30,800 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:30,800 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-27 03:54:30,800 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-27 03:54:30,802 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:30,802 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:30,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:30,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:30,803 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,803 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:30,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:30,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:30,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:30,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:30,805 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-27 03:54:30,805 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a32182f[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:30,805 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-27 03:54:30,805 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,807 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,807 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-27 03:54:30,807 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-27 03:54:30,807 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:30,807 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:30,807 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:30,809 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:30,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:30,809 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-27 03:54:30,809 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:30,809 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-27 03:54:30,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:30,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:30,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:30,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:30,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:30,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:30,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:30,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:30,816 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-27 03:54:30,817 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-27 03:54:30,818 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:30,818 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:30,818 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,818 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:30,820 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:30,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:30,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:30,821 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,821 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:30,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,821 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:30,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:30,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:30,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:30,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,823 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:30,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:30,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:30,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:30,825 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:30,825 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,825 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,825 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:30,826 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,826 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-27 03:54:30,826 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:30,826 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-27 03:54:30,826 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:30,826 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,826 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:30,826 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:30,826 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:30,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:30,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:30,826 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:30,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:30,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:30,927 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:30,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:30,932 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:30,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:30,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,934 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-27 03:54:30,934 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:30,935 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-27 03:54:30,935 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-27 03:54:30,935 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,935 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,936 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:30,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:30,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:30,936 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:30,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,937 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:30,937 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:30,938 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:30,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,939 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,939 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-27 03:54:30,939 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:30,939 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-27 03:54:30,939 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:30,939 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-27 03:54:30,940 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:30,940 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:30,940 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:30,940 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. started... 2023-03-27 03:54:30,940 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a03c85498f16edb6952ae7ec4494cfbd 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:31,354 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907f47237cfe2cf48dcbe0ed8a455e636cc to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20150907f47237cfe2cf48dcbe0ed8a455e636cc 2023-03-27 03:54:31,760 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/971634af027e481ea51d64a01cc77c68 2023-03-27 03:54:31,764 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/971634af027e481ea51d64a01cc77c68 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/971634af027e481ea51d64a01cc77c68 2023-03-27 03:54:31,768 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/971634af027e481ea51d64a01cc77c68, entries=1, sequenceid=37, filesize=5.0 K 2023-03-27 03:54:31,769 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a03c85498f16edb6952ae7ec4494cfbd in 829ms, sequenceid=37, compaction requested=true 2023-03-27 03:54:31,769 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:54:31,769 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:31,769 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:31,769 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:31,769 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:31,769 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-27 03:54:31,769 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-27 03:54:31,771 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,771 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:31,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:31,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:31,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:31,771 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,771 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:31,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:31,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:31,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:31,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:31,772 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-27 03:54:31,772 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31651538[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:31,772 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-27 03:54:31,772 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,773 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,774 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-27 03:54:31,774 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-27 03:54:31,774 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:31,774 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:31,774 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:31,775 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-27 03:54:31,775 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:31,775 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:31,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:31,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:31,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:31,775 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-27 03:54:31,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:31,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:31,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:31,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:31,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:31,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:31,777 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-27 03:54:31,777 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-27 03:54:31,777 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:31,777 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:31,777 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,777 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:31,779 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,779 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:31,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:31,779 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:31,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,780 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:31,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:31,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:31,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:31,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:31,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,781 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:31,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:31,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:31,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:31,783 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:31,783 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:31,783 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,783 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:31,783 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-27 03:54:31,783 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:31,783 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:31,783 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-03-27 03:54:31,783 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:31,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:31,783 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:31,783 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:31,783 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,783 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:31,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:31,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:31,783 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,784 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,883 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:31,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:31,888 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:31,889 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:31,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,890 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-27 03:54:31,890 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:31,890 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-03-27 03:54:31,890 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-03-27 03:54:31,890 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,890 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,893 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:31,893 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:31,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:31,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:31,894 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:31,894 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:31,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,894 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:31,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,894 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,895 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-03-27 03:54:31,895 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:31,895 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-03-27 03:54:31,895 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:31,895 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-03-27 03:54:31,895 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:31,895 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:31,895 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:31,895 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. started... 2023-03-27 03:54:31,895 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing a03c85498f16edb6952ae7ec4494cfbd 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:32,307 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120fad1e1c21a2b4e8b8a96f893dfc09b73 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120fad1e1c21a2b4e8b8a96f893dfc09b73 2023-03-27 03:54:32,713 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/c32404456df245c5892cd7e82239f5fd 2023-03-27 03:54:32,717 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/c32404456df245c5892cd7e82239f5fd as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/c32404456df245c5892cd7e82239f5fd 2023-03-27 03:54:32,721 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/c32404456df245c5892cd7e82239f5fd, entries=1, sequenceid=41, filesize=5.0 K 2023-03-27 03:54:32,722 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a03c85498f16edb6952ae7ec4494cfbd in 827ms, sequenceid=41, compaction requested=true 2023-03-27 03:54:32,722 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:54:32,722 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:32,722 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:32,722 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:32,722 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:32,722 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-03-27 03:54:32,722 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-03-27 03:54:32,724 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,724 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:32,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:32,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:32,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:32,724 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,724 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:32,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:32,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:32,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:32,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:32,726 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-03-27 03:54:32,726 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5483e6a0[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:32,726 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-03-27 03:54:32,726 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,727 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,727 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-03-27 03:54:32,727 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-03-27 03:54:32,727 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:32,727 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:32,727 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:32,728 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-03-27 03:54:32,728 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:32,728 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:32,728 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-03-27 03:54:32,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:32,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:32,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:32,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:32,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:32,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:32,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:32,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:32,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:32,730 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-03-27 03:54:32,730 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-03-27 03:54:32,730 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:32,730 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:32,730 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,730 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:32,733 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,733 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:32,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:32,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,733 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:32,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:32,733 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:32,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:32,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:32,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:32,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,734 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:32,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:32,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:32,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:32,736 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:32,737 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,737 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,737 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:32,737 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-03-27 03:54:32,737 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:32,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:32,737 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:32,737 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:32,737 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:32,737 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:32,737 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:32,737 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:32,737 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,737 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,737 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:32,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:32,837 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:32,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:32,839 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38271] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:32,841 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-27 03:54:32,841 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-27 03:54:32,842 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:54:32,842 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679889872842 2023-03-27 03:54:32,845 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-27 03:54:32,846 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-27 03:54:32,846 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-27 03:54:32,846 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] zookeeper.ReadOnlyZKClient(139): Connect 0x2ab3b343 to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:54:32,850 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3be9b372, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:54:32,850 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:33,686 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103eb47302d09054f41a94157a317fa554d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103eb47302d09054f41a94157a317fa554d 2023-03-27 03:54:33,687 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-27 03:54:33,699 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-27 03:54:33,700 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35010, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-27 03:54:33,707 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-27 03:54:33,709 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46344, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-27 03:54:33,718 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/49a6ee6d6a854471932c3febe1844eae first=Optional[r3] last=Optional[r6] 2023-03-27 03:54:33,719 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/49a6ee6d6a854471932c3febe1844eae}] 2023-03-27 03:54:33,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/49a6ee6d6a854471932c3febe1844eae for inclusion in a03c85498f16edb6952ae7ec4494cfbd/family1 2023-03-27 03:54:33,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-27 03:54:33,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last= 2023-03-27 03:54:33,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:54:33,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/49a6ee6d6a854471932c3febe1844eae to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__mp5gu9tpgp8allucsbvmmlifrvcvnsg2p5lot4n6lgqtibfv3akl5s0qss1r70bt/family1/49a6ee6d6a854471932c3febe1844eae 2023-03-27 03:54:33,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__mp5gu9tpgp8allucsbvmmlifrvcvnsg2p5lot4n6lgqtibfv3akl5s0qss1r70bt/family1/49a6ee6d6a854471932c3febe1844eae as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/7bc149bb3f4f4012858b762155b62730_SeqId_44_ 2023-03-27 03:54:33,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__mp5gu9tpgp8allucsbvmmlifrvcvnsg2p5lot4n6lgqtibfv3akl5s0qss1r70bt/family1/49a6ee6d6a854471932c3febe1844eae into a03c85498f16edb6952ae7ec4494cfbd/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/7bc149bb3f4f4012858b762155b62730_SeqId_44_ - updating store file list. 2023-03-27 03:54:33,733 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/7bc149bb3f4f4012858b762155b62730_SeqId_44_ into a03c85498f16edb6952ae7ec4494cfbd/family1 2023-03-27 03:54:33,733 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__mp5gu9tpgp8allucsbvmmlifrvcvnsg2p5lot4n6lgqtibfv3akl5s0qss1r70bt/family1/49a6ee6d6a854471932c3febe1844eae into a03c85498f16edb6952ae7ec4494cfbd/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/7bc149bb3f4f4012858b762155b62730_SeqId_44_) 2023-03-27 03:54:33,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__mp5gu9tpgp8allucsbvmmlifrvcvnsg2p5lot4n6lgqtibfv3akl5s0qss1r70bt/family1/49a6ee6d6a854471932c3febe1844eae 2023-03-27 03:54:33,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-27 03:54:33,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-27 03:54:33,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(1912): a03c85498f16edb6952ae7ec4494cfbd/family1 is initiating minor compaction 2023-03-27 03:54:33,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@8987a8e; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:33,737 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a03c85498f16edb6952ae7ec4494cfbd/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:33,737 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/b758b804dfab49a4983571f13ebb4b83, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/6062d443d71b49c590dfcde40f365b02, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/a2187447aca34853aaca274aa10e0fdb, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/29abc1b1b5f547aaa1084a394eb04bf2, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/337dffba0e4443388532170bb94328ac, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/a200694ff6c347ad8a0c0162bf758bf9, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/fb2143f4cab848a590a5998075925b40, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/971634af027e481ea51d64a01cc77c68, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/c32404456df245c5892cd7e82239f5fd, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/7bc149bb3f4f4012858b762155b62730_SeqId_44_] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp, totalSize=50.4 K 2023-03-27 03:54:33,737 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting b758b804dfab49a4983571f13ebb4b83, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-27 03:54:33,737 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 6062d443d71b49c590dfcde40f365b02, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-27 03:54:33,737 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting a2187447aca34853aaca274aa10e0fdb, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-27 03:54:33,738 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 29abc1b1b5f547aaa1084a394eb04bf2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-27 03:54:33,738 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 337dffba0e4443388532170bb94328ac, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-27 03:54:33,738 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting a200694ff6c347ad8a0c0162bf758bf9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-27 03:54:33,738 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting fb2143f4cab848a590a5998075925b40, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-27 03:54:33,738 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 971634af027e481ea51d64a01cc77c68, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-27 03:54:33,738 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting c32404456df245c5892cd7e82239f5fd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-27 03:54:33,738 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 7bc149bb3f4f4012858b762155b62730_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-27 03:54:33,739 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-27 03:54:33,739 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:33,741 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228e559b0197bfe4e5282943b1500fb1ba4 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228e559b0197bfe4e5282943b1500fb1ba4 2023-03-27 03:54:33,742 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015123106d65b6618634ec5a81875564fceeb16 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015123106d65b6618634ec5a81875564fceeb16 2023-03-27 03:54:33,743 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101a5a638b0ab7c405ebaed30957982aad9 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101a5a638b0ab7c405ebaed30957982aad9 2023-03-27 03:54:33,744 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601039b00c4bb01b74c9db8613301143bd724 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601039b00c4bb01b74c9db8613301143bd724 2023-03-27 03:54:33,744 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-03-27 03:54:33,745 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-27 03:54:33,745 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:33,745 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:33,745 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:33,745 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:33,745 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:33,745 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:33,745 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:33,745 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:33,760 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] throttle.PressureAwareThroughputController(145): a03c85498f16edb6952ae7ec4494cfbd#family1#compaction#61 average throughput is 0.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:54:34,575 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/e0b544c2ebca46c08165cd7c5737c686 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/e0b544c2ebca46c08165cd7c5737c686 2023-03-27 03:54:34,580 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in a03c85498f16edb6952ae7ec4494cfbd/family1 of a03c85498f16edb6952ae7ec4494cfbd into e0b544c2ebca46c08165cd7c5737c686(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:54:34,580 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:54:34,580 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1679889273737; duration=0sec 2023-03-27 03:54:34,580 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:34,582 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512056c9db819564947d88a048af63fe25b5d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512056c9db819564947d88a048af63fe25b5d 2023-03-27 03:54:34,582 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-27 03:54:34,595 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/b758b804dfab49a4983571f13ebb4b83, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/6062d443d71b49c590dfcde40f365b02, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/a2187447aca34853aaca274aa10e0fdb, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/29abc1b1b5f547aaa1084a394eb04bf2, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/337dffba0e4443388532170bb94328ac, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/a200694ff6c347ad8a0c0162bf758bf9, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/fb2143f4cab848a590a5998075925b40, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/971634af027e481ea51d64a01cc77c68, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/c32404456df245c5892cd7e82239f5fd, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/7bc149bb3f4f4012858b762155b62730_SeqId_44_] to archive 2023-03-27 03:54:34,596 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:34,598 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/b758b804dfab49a4983571f13ebb4b83 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/b758b804dfab49a4983571f13ebb4b83 2023-03-27 03:54:34,599 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/6062d443d71b49c590dfcde40f365b02 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/6062d443d71b49c590dfcde40f365b02 2023-03-27 03:54:34,601 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/a2187447aca34853aaca274aa10e0fdb to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/a2187447aca34853aaca274aa10e0fdb 2023-03-27 03:54:34,602 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/29abc1b1b5f547aaa1084a394eb04bf2 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/29abc1b1b5f547aaa1084a394eb04bf2 2023-03-27 03:54:34,603 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/337dffba0e4443388532170bb94328ac to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/337dffba0e4443388532170bb94328ac 2023-03-27 03:54:34,606 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/a200694ff6c347ad8a0c0162bf758bf9 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/a200694ff6c347ad8a0c0162bf758bf9 2023-03-27 03:54:34,607 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/fb2143f4cab848a590a5998075925b40 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/fb2143f4cab848a590a5998075925b40 2023-03-27 03:54:34,608 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/2942b3e60816489799421acd6c5348c5 first=Optional[r00] last=Optional[r2] 2023-03-27 03:54:34,608 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/971634af027e481ea51d64a01cc77c68 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/971634af027e481ea51d64a01cc77c68 2023-03-27 03:54:34,609 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/2942b3e60816489799421acd6c5348c5}] 2023-03-27 03:54:34,609 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/c32404456df245c5892cd7e82239f5fd to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/c32404456df245c5892cd7e82239f5fd 2023-03-27 03:54:34,610 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/2942b3e60816489799421acd6c5348c5 for inclusion in a03c85498f16edb6952ae7ec4494cfbd/family1 2023-03-27 03:54:34,611 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/7bc149bb3f4f4012858b762155b62730_SeqId_44_ to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/7bc149bb3f4f4012858b762155b62730_SeqId_44_ 2023-03-27 03:54:34,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-27 03:54:34,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last= 2023-03-27 03:54:34,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:54:34,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/2942b3e60816489799421acd6c5348c5 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__50k14h0bt0994u9j59m0vi7bslm3ubncnsntpc09voqr7drm8sm5dk0al1qcul3m/family1/2942b3e60816489799421acd6c5348c5 2023-03-27 03:54:34,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__50k14h0bt0994u9j59m0vi7bslm3ubncnsntpc09voqr7drm8sm5dk0al1qcul3m/family1/2942b3e60816489799421acd6c5348c5 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/196c9d369ce249e09a8e5216ff047556_SeqId_47_ 2023-03-27 03:54:34,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__50k14h0bt0994u9j59m0vi7bslm3ubncnsntpc09voqr7drm8sm5dk0al1qcul3m/family1/2942b3e60816489799421acd6c5348c5 into a03c85498f16edb6952ae7ec4494cfbd/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/196c9d369ce249e09a8e5216ff047556_SeqId_47_ - updating store file list. 2023-03-27 03:54:34,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/196c9d369ce249e09a8e5216ff047556_SeqId_47_ into a03c85498f16edb6952ae7ec4494cfbd/family1 2023-03-27 03:54:34,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__50k14h0bt0994u9j59m0vi7bslm3ubncnsntpc09voqr7drm8sm5dk0al1qcul3m/family1/2942b3e60816489799421acd6c5348c5 into a03c85498f16edb6952ae7ec4494cfbd/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/196c9d369ce249e09a8e5216ff047556_SeqId_47_) 2023-03-27 03:54:34,621 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__50k14h0bt0994u9j59m0vi7bslm3ubncnsntpc09voqr7drm8sm5dk0al1qcul3m/family1/2942b3e60816489799421acd6c5348c5 2023-03-27 03:54:34,621 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-27 03:54:34,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-27 03:54:34,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(1912): a03c85498f16edb6952ae7ec4494cfbd/family1 is initiating minor compaction (all files) 2023-03-27 03:54:34,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2e9aedc3; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:34,622 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a03c85498f16edb6952ae7ec4494cfbd/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:34,622 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/29fe9f0c28914c6d8f434d9e1c8da664, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/e0b544c2ebca46c08165cd7c5737c686, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/196c9d369ce249e09a8e5216ff047556_SeqId_47_] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp, totalSize=390.1 K 2023-03-27 03:54:34,622 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 29fe9f0c28914c6d8f434d9e1c8da664, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878842162 2023-03-27 03:54:34,623 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting e0b544c2ebca46c08165cd7c5737c686, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621242162 2023-03-27 03:54:34,623 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 196c9d369ce249e09a8e5216ff047556_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878842162 2023-03-27 03:54:34,623 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-27 03:54:34,624 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:34,631 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015113049acd9f8e9964b32a5d062ec09316966 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015113049acd9f8e9964b32a5d062ec09316966 2023-03-27 03:54:34,632 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201dc32484c31d24531865b3a8cc0eeb37a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201dc32484c31d24531865b3a8cc0eeb37a 2023-03-27 03:54:34,633 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205ac6762dc8d7a4d7cada9be949325d91f to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205ac6762dc8d7a4d7cada9be949325d91f 2023-03-27 03:54:34,634 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-03-27 03:54:34,634 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-27 03:54:34,634 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:34,634 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-27 03:54:34,634 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:54:34,635 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-27 03:54:34,637 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.7950 sec 2023-03-27 03:54:34,640 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] throttle.PressureAwareThroughputController(145): a03c85498f16edb6952ae7ec4494cfbd#family1#compaction#62 average throughput is 20.07 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:54:34,707 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-27 03:54:34,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:35,117 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-27 03:54:35,118 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-03-27 03:54:35,118 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-27 03:54:35,120 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-27 03:54:35,122 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a03c85498f16edb6952ae7ec4494cfbd, REOPEN/MOVE}] 2023-03-27 03:54:35,123 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a03c85498f16edb6952ae7ec4494cfbd, REOPEN/MOVE 2023-03-27 03:54:35,123 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=a03c85498f16edb6952ae7ec4494cfbd, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:35,123 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679889275123"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889275123"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889275123"}]},"ts":"1679889275123"} 2023-03-27 03:54:35,125 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure a03c85498f16edb6952ae7ec4494cfbd, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:54:35,276 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:35,276 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a03c85498f16edb6952ae7ec4494cfbd, disabling compactions & flushes 2023-03-27 03:54:35,276 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:35,856 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/.tmp/family1/67f9c965457f4d6dbb8d2d00f6a19512 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/67f9c965457f4d6dbb8d2d00f6a19512 2023-03-27 03:54:35,861 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in a03c85498f16edb6952ae7ec4494cfbd/family1 of a03c85498f16edb6952ae7ec4494cfbd into 67f9c965457f4d6dbb8d2d00f6a19512(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-27 03:54:35,861 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:54:35,861 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:35,861 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:35,861 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1679889274622; duration=1sec 2023-03-27 03:54:35,861 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. after waiting 0 ms 2023-03-27 03:54:35,861 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:35,861 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:35,861 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/29fe9f0c28914c6d8f434d9e1c8da664, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/e0b544c2ebca46c08165cd7c5737c686, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/196c9d369ce249e09a8e5216ff047556_SeqId_47_] to archive 2023-03-27 03:54:35,862 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:35,864 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/29fe9f0c28914c6d8f434d9e1c8da664 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/29fe9f0c28914c6d8f434d9e1c8da664 2023-03-27 03:54:35,865 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/e0b544c2ebca46c08165cd7c5737c686 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/e0b544c2ebca46c08165cd7c5737c686 2023-03-27 03:54:35,866 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/196c9d369ce249e09a8e5216ff047556_SeqId_47_ to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/196c9d369ce249e09a8e5216ff047556_SeqId_47_ 2023-03-27 03:54:35,869 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-27 03:54:35,870 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:35,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:54:35,871 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: a03c85498f16edb6952ae7ec4494cfbd to self. 2023-03-27 03:54:35,872 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:35,873 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=a03c85498f16edb6952ae7ec4494cfbd, regionState=CLOSED 2023-03-27 03:54:35,873 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679889275873"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889275873"}]},"ts":"1679889275873"} 2023-03-27 03:54:35,875 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-03-27 03:54:35,876 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure a03c85498f16edb6952ae7ec4494cfbd, server=jenkins-hbase4.apache.org,39727,1679889218560 in 749 msec 2023-03-27 03:54:35,876 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a03c85498f16edb6952ae7ec4494cfbd, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=true 2023-03-27 03:54:36,027 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=a03c85498f16edb6952ae7ec4494cfbd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:36,027 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679889276027"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889276027"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889276027"}]},"ts":"1679889276027"} 2023-03-27 03:54:36,029 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure a03c85498f16edb6952ae7ec4494cfbd, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:54:36,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:36,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a03c85498f16edb6952ae7ec4494cfbd, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.', STARTKEY => '', ENDKEY => ''} 2023-03-27 03:54:36,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:36,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:36,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:36,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:36,184 INFO [StoreOpener-a03c85498f16edb6952ae7ec4494cfbd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:36,185 DEBUG [StoreOpener-a03c85498f16edb6952ae7ec4494cfbd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1 2023-03-27 03:54:36,185 DEBUG [StoreOpener-a03c85498f16edb6952ae7ec4494cfbd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1 2023-03-27 03:54:36,186 INFO [StoreOpener-a03c85498f16edb6952ae7ec4494cfbd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a03c85498f16edb6952ae7ec4494cfbd columnFamilyName family1 2023-03-27 03:54:36,190 DEBUG [StoreOpener-a03c85498f16edb6952ae7ec4494cfbd-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/67f9c965457f4d6dbb8d2d00f6a19512 2023-03-27 03:54:36,190 INFO [StoreOpener-a03c85498f16edb6952ae7ec4494cfbd-1] regionserver.HStore(310): Store=a03c85498f16edb6952ae7ec4494cfbd/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:54:36,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:36,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:36,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:36,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a03c85498f16edb6952ae7ec4494cfbd; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68482878, jitterRate=0.020474404096603394}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-27 03:54:36,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:54:36,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd., pid=47, masterSystemTime=1679889276180 2023-03-27 03:54:36,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:36,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:36,198 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=a03c85498f16edb6952ae7ec4494cfbd, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:36,198 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679889276198"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889276198"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889276198"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889276198"}]},"ts":"1679889276198"} 2023-03-27 03:54:36,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-03-27 03:54:36,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure a03c85498f16edb6952ae7ec4494cfbd, server=jenkins-hbase4.apache.org,39727,1679889218560 in 170 msec 2023-03-27 03:54:36,202 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-03-27 03:54:36,202 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a03c85498f16edb6952ae7ec4494cfbd, REOPEN/MOVE in 1.0780 sec 2023-03-27 03:54:36,203 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-03-27 03:54:36,203 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.0820 sec 2023-03-27 03:54:36,205 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.4970 sec 2023-03-27 03:54:36,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-03-27 03:54:36,207 INFO [Listener at localhost/36131] compactions.TestMobCompactor(278): alter status finished 2023-03-27 03:54:36,207 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38271] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:36,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-27 03:54:36,209 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-27 03:54:36,210 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:54:36,210 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679889876210 2023-03-27 03:54:36,210 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-27 03:54:36,211 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-27 03:54:36,211 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-27 03:54:36,211 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] zookeeper.ReadOnlyZKClient(139): Connect 0x6d7463d6 to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:54:36,215 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1b702f29, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:54:36,215 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:36,215 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:36,215 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:36,215 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:36,215 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:37,040 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511281d6ea006d0ad49289e36c5482f6a973f to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511281d6ea006d0ad49289e36c5482f6a973f 2023-03-27 03:54:37,040 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-27 03:54:37,053 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-27 03:54:37,054 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35018, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-27 03:54:37,060 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-27 03:54:37,062 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46358, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-27 03:54:37,069 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/8e38737f8a0447909248c825d50981cc first=Optional[r05] last=Optional[r8] 2023-03-27 03:54:37,070 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=51 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/8e38737f8a0447909248c825d50981cc}] 2023-03-27 03:54:37,074 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/8e38737f8a0447909248c825d50981cc for inclusion in a03c85498f16edb6952ae7ec4494cfbd/family1 2023-03-27 03:54:37,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-27 03:54:37,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last= 2023-03-27 03:54:37,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:54:37,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/8e38737f8a0447909248c825d50981cc to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__j9r4nnkpbu3q6gou9tnp5vh8mds3qmn3mjq1o1u002n26clokq9h7t07ccjrkrle/family1/8e38737f8a0447909248c825d50981cc 2023-03-27 03:54:37,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__j9r4nnkpbu3q6gou9tnp5vh8mds3qmn3mjq1o1u002n26clokq9h7t07ccjrkrle/family1/8e38737f8a0447909248c825d50981cc as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/42ce47e629ac41f8b90953bf9eb0b7b0_SeqId_53_ 2023-03-27 03:54:37,080 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__j9r4nnkpbu3q6gou9tnp5vh8mds3qmn3mjq1o1u002n26clokq9h7t07ccjrkrle/family1/8e38737f8a0447909248c825d50981cc into a03c85498f16edb6952ae7ec4494cfbd/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/42ce47e629ac41f8b90953bf9eb0b7b0_SeqId_53_ - updating store file list. 2023-03-27 03:54:37,083 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/42ce47e629ac41f8b90953bf9eb0b7b0_SeqId_53_ into a03c85498f16edb6952ae7ec4494cfbd/family1 2023-03-27 03:54:37,083 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__j9r4nnkpbu3q6gou9tnp5vh8mds3qmn3mjq1o1u002n26clokq9h7t07ccjrkrle/family1/8e38737f8a0447909248c825d50981cc into a03c85498f16edb6952ae7ec4494cfbd/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/42ce47e629ac41f8b90953bf9eb0b7b0_SeqId_53_) 2023-03-27 03:54:37,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__j9r4nnkpbu3q6gou9tnp5vh8mds3qmn3mjq1o1u002n26clokq9h7t07ccjrkrle/family1/8e38737f8a0447909248c825d50981cc 2023-03-27 03:54:37,086 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-27 03:54:37,086 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:37,087 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120fad1e1c21a2b4e8b8a96f893dfc09b73 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120fad1e1c21a2b4e8b8a96f893dfc09b73 2023-03-27 03:54:37,088 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511289db2ad7cca3249d188f6cdfddbb920ee to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511289db2ad7cca3249d188f6cdfddbb920ee 2023-03-27 03:54:37,088 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-03-27 03:54:37,088 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-27 03:54:37,089 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:37,089 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:37,089 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:37,089 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-27 03:54:37,089 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:54:37,090 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-27 03:54:37,091 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 882 msec 2023-03-27 03:54:37,134 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-27 03:54:37,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:37,547 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-27 03:54:37,548 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-03-27 03:54:37,549 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-03-27 03:54:37,551 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-03-27 03:54:37,554 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a03c85498f16edb6952ae7ec4494cfbd, REOPEN/MOVE}] 2023-03-27 03:54:37,555 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a03c85498f16edb6952ae7ec4494cfbd, REOPEN/MOVE 2023-03-27 03:54:37,555 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=a03c85498f16edb6952ae7ec4494cfbd, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:37,556 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679889277555"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889277555"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889277555"}]},"ts":"1679889277555"} 2023-03-27 03:54:37,557 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure a03c85498f16edb6952ae7ec4494cfbd, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:54:37,709 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:37,709 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a03c85498f16edb6952ae7ec4494cfbd, disabling compactions & flushes 2023-03-27 03:54:37,709 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:37,709 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:37,709 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. after waiting 0 ms 2023-03-27 03:54:37,709 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:37,713 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-27 03:54:37,713 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:37,713 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:54:37,714 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: a03c85498f16edb6952ae7ec4494cfbd to self. 2023-03-27 03:54:37,715 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:37,715 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=a03c85498f16edb6952ae7ec4494cfbd, regionState=CLOSED 2023-03-27 03:54:37,715 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679889277715"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889277715"}]},"ts":"1679889277715"} 2023-03-27 03:54:37,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-03-27 03:54:37,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure a03c85498f16edb6952ae7ec4494cfbd, server=jenkins-hbase4.apache.org,39727,1679889218560 in 159 msec 2023-03-27 03:54:37,718 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a03c85498f16edb6952ae7ec4494cfbd, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=true 2023-03-27 03:54:37,869 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=a03c85498f16edb6952ae7ec4494cfbd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:37,869 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679889277869"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889277869"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889277869"}]},"ts":"1679889277869"} 2023-03-27 03:54:37,871 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure a03c85498f16edb6952ae7ec4494cfbd, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:54:38,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:38,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a03c85498f16edb6952ae7ec4494cfbd, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.', STARTKEY => '', ENDKEY => ''} 2023-03-27 03:54:38,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:38,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:38,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:38,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:38,026 INFO [StoreOpener-a03c85498f16edb6952ae7ec4494cfbd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:38,027 DEBUG [StoreOpener-a03c85498f16edb6952ae7ec4494cfbd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1 2023-03-27 03:54:38,027 DEBUG [StoreOpener-a03c85498f16edb6952ae7ec4494cfbd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1 2023-03-27 03:54:38,027 INFO [StoreOpener-a03c85498f16edb6952ae7ec4494cfbd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a03c85498f16edb6952ae7ec4494cfbd columnFamilyName family1 2023-03-27 03:54:38,032 DEBUG [StoreOpener-a03c85498f16edb6952ae7ec4494cfbd-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/42ce47e629ac41f8b90953bf9eb0b7b0_SeqId_53_ 2023-03-27 03:54:38,034 DEBUG [StoreOpener-a03c85498f16edb6952ae7ec4494cfbd-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/family1/67f9c965457f4d6dbb8d2d00f6a19512 2023-03-27 03:54:38,035 INFO [StoreOpener-a03c85498f16edb6952ae7ec4494cfbd-1] regionserver.HStore(310): Store=a03c85498f16edb6952ae7ec4494cfbd/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:54:38,035 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:38,036 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:38,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:54:38,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a03c85498f16edb6952ae7ec4494cfbd; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62028698, jitterRate=-0.07570037245750427}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-27 03:54:38,039 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:54:38,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd., pid=53, masterSystemTime=1679889278022 2023-03-27 03:54:38,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:38,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:54:38,041 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=a03c85498f16edb6952ae7ec4494cfbd, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:38,042 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1679889278041"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889278041"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889278041"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889278041"}]},"ts":"1679889278041"} 2023-03-27 03:54:38,044 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-03-27 03:54:38,044 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure a03c85498f16edb6952ae7ec4494cfbd, server=jenkins-hbase4.apache.org,39727,1679889218560 in 173 msec 2023-03-27 03:54:38,045 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-03-27 03:54:38,045 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=a03c85498f16edb6952ae7ec4494cfbd, REOPEN/MOVE in 490 msec 2023-03-27 03:54:38,047 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-03-27 03:54:38,047 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 495 msec 2023-03-27 03:54:38,048 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 913 msec 2023-03-27 03:54:38,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-03-27 03:54:38,050 INFO [Listener at localhost/36131] compactions.TestMobCompactor(278): alter status finished 2023-03-27 03:54:38,051 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38271] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-03-27 03:54:38,052 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-27 03:54:38,052 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-27 03:54:38,053 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:54:38,053 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679889878053 2023-03-27 03:54:38,053 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-27 03:54:38,054 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-03-27 03:54:38,054 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-27 03:54:38,054 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] zookeeper.ReadOnlyZKClient(139): Connect 0x20edfee5 to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:54:38,059 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3afe94b6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:54:38,059 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:38,059 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:38,059 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:38,059 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:38,059 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:38,059 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:38,059 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:38,059 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-03-27 03:54:38,059 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-03-27 03:54:38,059 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:54:38,060 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-03-27 03:54:38,061 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 9 msec 2023-03-27 03:54:38,106 INFO [Listener at localhost/36131] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=256 (was 266), OpenFileDescriptor=632 (was 637), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=437 (was 470), ProcessCount=170 (was 170), AvailableMemoryMB=8808 (was 8805) - AvailableMemoryMB LEAK? - 2023-03-27 03:54:38,117 INFO [Listener at localhost/36131] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=256, OpenFileDescriptor=632, MaxFileDescriptor=60000, SystemLoadAverage=437, ProcessCount=170, AvailableMemoryMB=8807 2023-03-27 03:54:38,118 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-03-27 03:54:38,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:38,120 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-03-27 03:54:38,120 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-03-27 03:54:38,121 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-27 03:54:38,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-27 03:54:38,122 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:38,122 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753 empty. 2023-03-27 03:54:38,123 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:38,123 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-03-27 03:54:38,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-27 03:54:38,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-27 03:54:38,530 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-27 03:54:38,531 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7675): creating {ENCODED => 1b540ab2fb7dd087d0e66746ccea8753, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp 2023-03-27 03:54:38,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-27 03:54:38,937 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:38,937 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1604): Closing 1b540ab2fb7dd087d0e66746ccea8753, disabling compactions & flushes 2023-03-27 03:54:38,937 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:38,937 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:38,937 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. after waiting 0 ms 2023-03-27 03:54:38,937 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:38,937 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:38,937 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1558): Region close journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:54:38,940 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-03-27 03:54:38,941 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679889278941"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889278941"}]},"ts":"1679889278941"} 2023-03-27 03:54:38,942 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-27 03:54:38,943 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-27 03:54:38,943 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889278943"}]},"ts":"1679889278943"} 2023-03-27 03:54:38,943 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-03-27 03:54:38,946 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1b540ab2fb7dd087d0e66746ccea8753, ASSIGN}] 2023-03-27 03:54:38,947 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1b540ab2fb7dd087d0e66746ccea8753, ASSIGN 2023-03-27 03:54:38,948 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1b540ab2fb7dd087d0e66746ccea8753, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=false 2023-03-27 03:54:39,099 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=1b540ab2fb7dd087d0e66746ccea8753, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:39,099 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679889279099"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889279099"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889279099"}]},"ts":"1679889279099"} 2023-03-27 03:54:39,101 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure 1b540ab2fb7dd087d0e66746ccea8753, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:54:39,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-27 03:54:39,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:39,255 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1b540ab2fb7dd087d0e66746ccea8753, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753.', STARTKEY => '', ENDKEY => ''} 2023-03-27 03:54:39,255 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:39,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:39,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:39,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:39,257 INFO [StoreOpener-1b540ab2fb7dd087d0e66746ccea8753-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:39,258 DEBUG [StoreOpener-1b540ab2fb7dd087d0e66746ccea8753-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1 2023-03-27 03:54:39,258 DEBUG [StoreOpener-1b540ab2fb7dd087d0e66746ccea8753-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1 2023-03-27 03:54:39,258 INFO [StoreOpener-1b540ab2fb7dd087d0e66746ccea8753-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b540ab2fb7dd087d0e66746ccea8753 columnFamilyName family1 2023-03-27 03:54:39,260 INFO [StoreOpener-1b540ab2fb7dd087d0e66746ccea8753-1] regionserver.HStore(310): Store=1b540ab2fb7dd087d0e66746ccea8753/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:54:39,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:39,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:39,263 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:39,265 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:54:39,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1b540ab2fb7dd087d0e66746ccea8753; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69448945, jitterRate=0.03486992418766022}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-27 03:54:39,265 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:54:39,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753., pid=57, masterSystemTime=1679889279253 2023-03-27 03:54:39,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:39,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:39,268 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=1b540ab2fb7dd087d0e66746ccea8753, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:39,268 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679889279268"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889279268"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889279268"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889279268"}]},"ts":"1679889279268"} 2023-03-27 03:54:39,271 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-03-27 03:54:39,271 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure 1b540ab2fb7dd087d0e66746ccea8753, server=jenkins-hbase4.apache.org,39727,1679889218560 in 168 msec 2023-03-27 03:54:39,272 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-03-27 03:54:39,272 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1b540ab2fb7dd087d0e66746ccea8753, ASSIGN in 325 msec 2023-03-27 03:54:39,273 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-27 03:54:39,273 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889279273"}]},"ts":"1679889279273"} 2023-03-27 03:54:39,274 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-03-27 03:54:39,276 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-03-27 03:54:39,277 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1580 sec 2023-03-27 03:54:40,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-03-27 03:54:40,226 INFO [Listener at localhost/36131] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-03-27 03:54:40,244 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:40,246 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:40,246 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-27 03:54:40,246 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:40,246 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-27 03:54:40,246 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-27 03:54:40,246 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:40,247 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:40,248 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:40,248 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:40,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:40,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:40,248 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:40,248 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:40,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:40,248 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:40,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:40,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:40,249 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:40,250 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:40,250 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-27 03:54:40,250 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:40,250 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-27 03:54:40,250 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:40,251 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:40,251 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:40,251 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. started... 2023-03-27 03:54:40,251 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1b540ab2fb7dd087d0e66746ccea8753 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-27 03:54:40,290 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-27 03:54:40,676 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113033112232b0754428b1e80b6e0e98fa76 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015113033112232b0754428b1e80b6e0e98fa76 2023-03-27 03:54:40,868 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-27 03:54:41,080 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/b4d440078d12497fa79327245c339db4 2023-03-27 03:54:41,084 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/b4d440078d12497fa79327245c339db4 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/b4d440078d12497fa79327245c339db4 2023-03-27 03:54:41,088 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/b4d440078d12497fa79327245c339db4, entries=1000, sequenceid=5, filesize=190.5 K 2023-03-27 03:54:41,088 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 1b540ab2fb7dd087d0e66746ccea8753 in 837ms, sequenceid=5, compaction requested=false 2023-03-27 03:54:41,089 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:54:41,089 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:41,089 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:41,089 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:41,089 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:41,089 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-27 03:54:41,089 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-27 03:54:41,091 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:41,091 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:41,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:41,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:41,091 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,091 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:41,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:41,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:41,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:41,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:41,092 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-27 03:54:41,092 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5eb5c201[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:41,092 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-27 03:54:41,093 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,095 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,095 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-27 03:54:41,095 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-27 03:54:41,095 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:41,095 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:41,095 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:41,096 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-27 03:54:41,096 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:41,096 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:41,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:41,096 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-27 03:54:41,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:41,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:41,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:41,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:41,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:41,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:41,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:41,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:41,098 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-27 03:54:41,098 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-27 03:54:41,098 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:41,098 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:41,098 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,098 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:41,099 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,099 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:41,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:41,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,100 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:41,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:41,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:41,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:41,100 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:41,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:41,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:41,101 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:41,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:41,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:41,108 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:41,108 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:41,108 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,108 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:41,108 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:41,108 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-27 03:54:41,108 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:41,108 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:41,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:41,108 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:41,108 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,108 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,108 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:41,108 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:41,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:41,108 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:41,208 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:41,209 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:41,213 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:41,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:41,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,215 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-27 03:54:41,215 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:41,215 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-27 03:54:41,215 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-27 03:54:41,215 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,215 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,216 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:41,216 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:41,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:41,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:41,216 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:41,216 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:41,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,217 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:41,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,217 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,217 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-27 03:54:41,218 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:41,218 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-27 03:54:41,218 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:41,218 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-27 03:54:41,218 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:41,218 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:41,218 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:41,218 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. started... 2023-03-27 03:54:41,218 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1b540ab2fb7dd087d0e66746ccea8753 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-27 03:54:41,399 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-27 03:54:41,634 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128f2ece1c400c6449697adf1cd5b066492 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128f2ece1c400c6449697adf1cd5b066492 2023-03-27 03:54:42,039 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/fab57fbeb0eb4ebb9c707e0c187c1032 2023-03-27 03:54:42,044 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/fab57fbeb0eb4ebb9c707e0c187c1032 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/fab57fbeb0eb4ebb9c707e0c187c1032 2023-03-27 03:54:42,047 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/fab57fbeb0eb4ebb9c707e0c187c1032, entries=1, sequenceid=9, filesize=5.0 K 2023-03-27 03:54:42,048 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 1b540ab2fb7dd087d0e66746ccea8753 in 830ms, sequenceid=9, compaction requested=false 2023-03-27 03:54:42,048 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:54:42,048 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:42,048 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:42,048 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:42,048 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:42,048 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-27 03:54:42,048 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-27 03:54:42,051 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,051 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:42,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:42,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:42,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:42,051 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,051 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:42,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:42,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:42,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:42,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:42,052 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-27 03:54:42,052 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@342c4a98[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:42,052 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-27 03:54:42,052 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,053 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,054 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-27 03:54:42,054 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-27 03:54:42,054 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:42,054 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:42,054 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:42,055 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-27 03:54:42,055 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:42,055 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:42,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:42,055 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-27 03:54:42,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:42,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:42,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:42,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:42,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:42,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:42,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:42,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:42,057 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-27 03:54:42,057 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-27 03:54:42,057 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:42,057 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:42,057 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,057 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:42,058 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,058 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:42,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:42,058 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:42,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:42,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:42,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:42,059 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:42,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:42,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:42,060 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:42,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:42,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:42,062 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:42,062 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,062 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,062 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:42,062 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-27 03:54:42,062 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:42,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:42,062 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:42,062 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:42,062 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:42,063 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:42,063 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:42,063 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,063 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:42,063 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:42,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:42,163 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:42,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:42,168 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:42,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:42,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,169 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-27 03:54:42,169 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:42,169 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-27 03:54:42,169 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-27 03:54:42,170 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,170 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,171 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:42,171 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:42,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:42,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:42,171 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:42,171 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:42,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,171 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:42,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,172 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,172 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-27 03:54:42,172 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:42,172 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-27 03:54:42,172 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:42,172 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-27 03:54:42,172 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:42,172 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:42,172 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:42,172 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. started... 2023-03-27 03:54:42,173 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1b540ab2fb7dd087d0e66746ccea8753 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:42,584 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512015f7f06c5482445eebca24cc8dffcd7a4 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512015f7f06c5482445eebca24cc8dffcd7a4 2023-03-27 03:54:42,990 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/fddf8d78bdb94f0ebcf0e33af2814806 2023-03-27 03:54:42,995 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/fddf8d78bdb94f0ebcf0e33af2814806 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/fddf8d78bdb94f0ebcf0e33af2814806 2023-03-27 03:54:42,999 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/fddf8d78bdb94f0ebcf0e33af2814806, entries=1, sequenceid=13, filesize=5.0 K 2023-03-27 03:54:42,999 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1b540ab2fb7dd087d0e66746ccea8753 in 827ms, sequenceid=13, compaction requested=true 2023-03-27 03:54:43,000 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:54:43,000 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:43,000 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:43,000 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:43,000 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,000 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-27 03:54:43,000 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-27 03:54:43,001 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,001 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:43,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:43,002 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,002 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:43,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:43,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:43,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:43,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-27 03:54:43,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2df945f0[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:43,003 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-27 03:54:43,003 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,004 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,004 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-27 03:54:43,004 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-27 03:54:43,004 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:43,004 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,004 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:43,006 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-27 03:54:43,006 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,006 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:43,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:43,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:43,006 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-27 03:54:43,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:43,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:43,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:43,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:43,009 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-27 03:54:43,009 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-27 03:54:43,009 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:43,009 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:43,009 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,009 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:43,010 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,010 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,010 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:43,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:43,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:43,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:43,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:43,011 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:43,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:43,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,012 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:43,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,013 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,014 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:43,014 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:43,014 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:43,014 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-27 03:54:43,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:43,014 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,014 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:43,014 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,014 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:43,014 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,014 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:43,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:43,014 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,014 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:43,014 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:43,114 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:43,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:43,119 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:43,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:43,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,121 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-27 03:54:43,121 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:43,121 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-27 03:54:43,121 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-27 03:54:43,121 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,121 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,123 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,123 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:43,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:43,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:43,123 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,123 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:43,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,123 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:43,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,124 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,124 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-27 03:54:43,124 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,124 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-27 03:54:43,124 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:43,124 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-27 03:54:43,124 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:43,124 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:43,124 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:43,124 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. started... 2023-03-27 03:54:43,124 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1b540ab2fb7dd087d0e66746ccea8753 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:43,537 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205efddd440f042426e92add508e370c1ac to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205efddd440f042426e92add508e370c1ac 2023-03-27 03:54:43,947 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/489d2ef3acda4336b260abe1ef6414d4 2023-03-27 03:54:43,951 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/489d2ef3acda4336b260abe1ef6414d4 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/489d2ef3acda4336b260abe1ef6414d4 2023-03-27 03:54:43,954 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/489d2ef3acda4336b260abe1ef6414d4, entries=1, sequenceid=17, filesize=5.0 K 2023-03-27 03:54:43,955 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1b540ab2fb7dd087d0e66746ccea8753 in 831ms, sequenceid=17, compaction requested=true 2023-03-27 03:54:43,955 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:54:43,955 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:43,955 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:43,955 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:43,955 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,955 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-27 03:54:43,955 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-27 03:54:43,957 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,957 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:43,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:43,957 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,957 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:43,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:43,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:43,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:43,958 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-27 03:54:43,958 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66840009[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:43,958 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-27 03:54:43,958 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,965 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,965 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-27 03:54:43,965 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-27 03:54:43,965 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:43,965 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,965 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:43,966 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-27 03:54:43,966 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,966 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:43,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:43,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:43,966 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-27 03:54:43,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:43,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:43,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:43,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:43,968 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-27 03:54:43,968 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-27 03:54:43,968 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:43,968 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:43,968 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,968 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:43,969 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,969 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,970 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:43,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:43,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:43,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:43,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:43,970 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:43,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:43,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,971 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:43,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,973 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,973 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,973 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:43,973 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:43,973 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-27 03:54:43,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:43,973 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:43,973 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-27 03:54:43,973 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:43,973 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,973 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:43,973 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,973 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:43,973 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:43,974 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:43,973 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:43,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:43,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:44,074 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:44,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:44,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:44,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:44,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,080 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-27 03:54:44,080 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:44,080 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-27 03:54:44,080 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-27 03:54:44,080 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,080 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,082 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:44,082 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:44,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:44,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:44,083 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:44,083 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:44,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,083 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:44,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,083 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,084 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-27 03:54:44,084 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,084 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-27 03:54:44,084 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:44,084 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-27 03:54:44,084 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:44,084 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:44,084 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:44,084 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. started... 2023-03-27 03:54:44,084 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1b540ab2fb7dd087d0e66746ccea8753 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:44,496 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122840f1ca9b7a0544e181e01c543e90ff0a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015122840f1ca9b7a0544e181e01c543e90ff0a 2023-03-27 03:54:44,902 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/879be0ab998b457194fe0279e687ad85 2023-03-27 03:54:44,907 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/879be0ab998b457194fe0279e687ad85 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/879be0ab998b457194fe0279e687ad85 2023-03-27 03:54:44,910 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/879be0ab998b457194fe0279e687ad85, entries=1, sequenceid=21, filesize=5.0 K 2023-03-27 03:54:44,911 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1b540ab2fb7dd087d0e66746ccea8753 in 827ms, sequenceid=21, compaction requested=true 2023-03-27 03:54:44,911 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:54:44,911 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:44,911 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:44,911 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:44,911 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:44,911 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-27 03:54:44,911 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-27 03:54:44,914 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,914 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:44,914 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,914 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:44,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:44,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:44,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:44,915 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-27 03:54:44,915 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16516312[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:44,915 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-27 03:54:44,915 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,916 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,916 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-27 03:54:44,916 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-27 03:54:44,916 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:44,916 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:44,916 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:44,917 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:44,917 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-27 03:54:44,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:44,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:44,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:44,917 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:44,918 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-27 03:54:44,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:44,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:44,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:44,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:44,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:44,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:44,919 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-27 03:54:44,919 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-27 03:54:44,919 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:44,919 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:44,919 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,919 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:44,921 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,921 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:44,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:44,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,921 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:44,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:44,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:44,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:44,921 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:44,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:44,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,922 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:44,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:44,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:44,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:44,928 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:44,928 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:44,928 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:44,928 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,928 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:44,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:44,928 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:44,928 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-27 03:54:44,929 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:44,928 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:44,929 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,929 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,929 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:44,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:44,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:44,929 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:44,929 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:45,029 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:45,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:45,034 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:45,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:45,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,035 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-27 03:54:45,035 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:45,035 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-27 03:54:45,035 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-27 03:54:45,036 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,036 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,037 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:45,037 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:45,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:45,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:45,037 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:45,037 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:45,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,038 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:45,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,038 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,038 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-27 03:54:45,038 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,038 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-27 03:54:45,038 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:45,038 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-27 03:54:45,039 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:45,039 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:45,039 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:45,039 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. started... 2023-03-27 03:54:45,039 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1b540ab2fb7dd087d0e66746ccea8753 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:45,450 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231718504109a1c45d1b43fa65169b99021 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231718504109a1c45d1b43fa65169b99021 2023-03-27 03:54:45,855 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/07cebda199ec4e349efbd8b96ca98d62 2023-03-27 03:54:45,861 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/07cebda199ec4e349efbd8b96ca98d62 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/07cebda199ec4e349efbd8b96ca98d62 2023-03-27 03:54:45,866 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/07cebda199ec4e349efbd8b96ca98d62, entries=1, sequenceid=25, filesize=5.0 K 2023-03-27 03:54:45,867 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1b540ab2fb7dd087d0e66746ccea8753 in 828ms, sequenceid=25, compaction requested=true 2023-03-27 03:54:45,867 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:54:45,867 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:45,867 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:45,868 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:45,868 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:45,868 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-27 03:54:45,868 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-27 03:54:45,873 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,873 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:45,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:45,874 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,874 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:45,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:45,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:45,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:45,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:45,875 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-27 03:54:45,876 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@40ff93ca[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:45,876 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-27 03:54:45,876 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,877 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,877 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-27 03:54:45,877 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-27 03:54:45,877 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:45,877 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:45,877 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:45,879 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-27 03:54:45,879 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:45,879 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:45,879 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-27 03:54:45,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:45,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:45,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:45,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:45,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:45,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:45,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:45,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:45,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:45,882 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-27 03:54:45,882 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-27 03:54:45,882 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:45,882 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:45,882 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,882 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:45,886 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,886 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:45,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,887 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:45,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:45,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:45,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:45,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:45,888 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:45,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:45,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:45,890 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:45,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:45,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:45,897 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:45,897 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:45,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:45,898 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,897 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:45,898 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,898 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:45,898 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,898 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,898 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:45,898 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:45,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:45,898 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-27 03:54:45,898 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:45,899 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:45,899 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:45,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:45,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:45,999 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:46,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:46,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:46,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:46,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,008 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-27 03:54:46,008 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:46,008 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-27 03:54:46,008 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-27 03:54:46,009 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,009 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,018 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,018 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:46,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:46,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:46,018 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,018 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:46,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,019 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:46,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,020 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,020 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-27 03:54:46,020 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,020 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-27 03:54:46,020 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:46,020 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-27 03:54:46,021 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:46,021 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:46,021 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:46,021 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. started... 2023-03-27 03:54:46,021 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1b540ab2fb7dd087d0e66746ccea8753 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:46,072 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601019a9cf63dd7c44bf6910904f545d91cb9 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601019a9cf63dd7c44bf6910904f545d91cb9 2023-03-27 03:54:46,490 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/2b222396699f41b382f9800b311da70b 2023-03-27 03:54:46,495 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/2b222396699f41b382f9800b311da70b as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/2b222396699f41b382f9800b311da70b 2023-03-27 03:54:46,501 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/2b222396699f41b382f9800b311da70b, entries=1, sequenceid=29, filesize=5.0 K 2023-03-27 03:54:46,502 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1b540ab2fb7dd087d0e66746ccea8753 in 481ms, sequenceid=29, compaction requested=true 2023-03-27 03:54:46,502 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:54:46,502 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:46,502 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:46,502 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:46,502 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,502 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-27 03:54:46,502 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-27 03:54:46,504 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,504 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:46,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:46,504 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:46,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:46,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,504 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:46,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:46,506 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-27 03:54:46,506 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@274e98e0[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:46,508 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-27 03:54:46,508 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,510 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,510 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-27 03:54:46,510 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-27 03:54:46,510 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:46,510 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,510 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:46,512 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:46,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:46,512 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-27 03:54:46,512 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:46,512 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-27 03:54:46,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:46,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:46,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:46,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:46,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-27 03:54:46,516 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-27 03:54:46,516 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:46,516 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:46,516 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,516 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:46,517 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,517 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:46,517 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:46,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:46,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:46,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:46,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:46,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,518 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:46,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,520 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:46,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,523 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:46,523 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:46,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:46,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:46,524 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,524 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,524 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,524 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-27 03:54:46,524 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:46,524 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:46,524 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-27 03:54:46,524 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:46,524 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:46,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:46,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:46,524 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,524 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,525 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,525 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,624 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:46,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:46,630 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:46,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:46,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,633 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-27 03:54:46,633 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:46,633 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-27 03:54:46,633 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-27 03:54:46,633 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,633 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,635 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:46,635 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:46,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:46,635 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,635 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:46,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:46,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,636 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,636 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-27 03:54:46,636 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,636 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-27 03:54:46,636 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:46,636 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-27 03:54:46,637 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:46,637 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:46,637 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:46,637 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. started... 2023-03-27 03:54:46,637 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1b540ab2fb7dd087d0e66746ccea8753 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:46,690 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103324459e088f846938407ad538b58061b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103324459e088f846938407ad538b58061b 2023-03-27 03:54:46,704 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/9030953197314aeabbb90b18ecff4b9b 2023-03-27 03:54:46,709 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/9030953197314aeabbb90b18ecff4b9b as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/9030953197314aeabbb90b18ecff4b9b 2023-03-27 03:54:46,715 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/9030953197314aeabbb90b18ecff4b9b, entries=1, sequenceid=33, filesize=5.0 K 2023-03-27 03:54:46,716 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1b540ab2fb7dd087d0e66746ccea8753 in 79ms, sequenceid=33, compaction requested=true 2023-03-27 03:54:46,716 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:54:46,716 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:46,716 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:46,717 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:46,717 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,717 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-27 03:54:46,717 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-27 03:54:46,718 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,719 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:46,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:46,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:46,719 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,720 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:46,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:46,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:46,722 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-27 03:54:46,722 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@618c4048[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:46,722 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-27 03:54:46,722 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,724 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,724 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-27 03:54:46,725 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-27 03:54:46,725 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:46,725 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,725 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:46,733 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-27 03:54:46,733 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:46,733 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-27 03:54:46,733 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:46,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:46,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:46,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:46,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:46,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:46,738 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-27 03:54:46,738 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-27 03:54:46,738 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:46,738 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:46,738 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,738 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:46,740 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,740 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,740 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:46,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:46,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:46,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:46,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:46,741 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:46,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:46,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,742 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:46,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,746 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,746 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,746 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,746 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,746 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:46,746 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:46,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:46,746 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-27 03:54:46,746 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:46,746 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,746 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,746 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:46,746 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:46,746 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:46,746 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:46,746 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-27 03:54:46,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:46,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:46,847 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:46,848 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:46,852 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:46,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:46,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,854 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-27 03:54:46,854 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:46,854 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-27 03:54:46,854 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-27 03:54:46,854 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,854 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,855 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:46,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:46,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:46,855 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,856 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,856 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:46,856 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:46,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,856 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,857 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-27 03:54:46,857 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,857 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-27 03:54:46,857 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:46,857 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-27 03:54:46,857 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:46,857 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:46,857 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:46,857 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. started... 2023-03-27 03:54:46,857 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1b540ab2fb7dd087d0e66746ccea8753 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:46,887 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907b9a4e1f9c7cd4263b5428e48ff0f6ead to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20150907b9a4e1f9c7cd4263b5428e48ff0f6ead 2023-03-27 03:54:46,923 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/897e4c95464743f89a952b90d38b9608 2023-03-27 03:54:46,931 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/897e4c95464743f89a952b90d38b9608 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/897e4c95464743f89a952b90d38b9608 2023-03-27 03:54:46,939 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/897e4c95464743f89a952b90d38b9608, entries=1, sequenceid=37, filesize=5.0 K 2023-03-27 03:54:46,940 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1b540ab2fb7dd087d0e66746ccea8753 in 83ms, sequenceid=37, compaction requested=true 2023-03-27 03:54:46,940 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:54:46,940 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:46,940 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:46,941 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:46,941 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,941 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-27 03:54:46,941 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-27 03:54:46,943 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,943 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:46,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:46,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:46,943 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,943 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:46,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:46,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:46,950 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-27 03:54:46,950 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c58e698[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:46,950 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-27 03:54:46,950 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,952 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,952 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,952 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-27 03:54:46,952 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-27 03:54:46,952 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:46,953 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,953 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:46,955 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-27 03:54:46,955 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:46,955 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-27 03:54:46,955 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:46,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:46,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:46,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:46,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:46,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:46,963 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-27 03:54:46,963 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-27 03:54:46,963 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:46,963 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:46,963 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,963 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:46,965 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,965 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:46,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,966 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:46,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:46,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:46,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:46,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:46,967 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:46,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,969 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:46,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,974 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:46,974 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:46,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:46,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:46,974 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,974 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,974 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,974 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:46,974 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,974 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:46,974 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-27 03:54:46,975 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-03-27 03:54:46,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:46,975 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:46,975 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:46,975 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:46,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:46,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:47,075 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:47,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:47,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:47,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:47,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,084 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-27 03:54:47,084 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:47,084 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-03-27 03:54:47,084 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-03-27 03:54:47,085 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,085 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,086 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:47,086 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:47,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:47,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:47,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,087 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:47,087 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:47,088 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:47,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,089 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,089 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-03-27 03:54:47,089 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,089 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-03-27 03:54:47,089 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:47,089 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-03-27 03:54:47,092 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:54:47,092 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:47,092 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:47,092 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. started... 2023-03-27 03:54:47,092 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 1b540ab2fb7dd087d0e66746ccea8753 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:54:47,139 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120b5378f1072454231bbb3ff25931388a6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120b5378f1072454231bbb3ff25931388a6 2023-03-27 03:54:47,562 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/090c94ee37ee41bda6843ba9ed02549a 2023-03-27 03:54:47,568 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/090c94ee37ee41bda6843ba9ed02549a as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/090c94ee37ee41bda6843ba9ed02549a 2023-03-27 03:54:47,582 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/090c94ee37ee41bda6843ba9ed02549a, entries=1, sequenceid=41, filesize=5.0 K 2023-03-27 03:54:47,583 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1b540ab2fb7dd087d0e66746ccea8753 in 491ms, sequenceid=41, compaction requested=true 2023-03-27 03:54:47,584 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:54:47,584 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:47,584 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:54:47,584 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:54:47,584 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:47,584 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-03-27 03:54:47,584 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-03-27 03:54:47,586 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:47,586 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:47,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:47,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:47,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:47,586 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,586 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:47,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:47,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:47,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:47,588 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-03-27 03:54:47,588 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@40054b1a[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:47,588 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-03-27 03:54:47,588 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,590 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,591 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-03-27 03:54:47,591 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-03-27 03:54:47,591 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:47,591 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:47,591 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:47,593 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-03-27 03:54:47,593 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:47,593 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-03-27 03:54:47,593 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:47,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:47,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:47,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:47,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:47,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:47,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:47,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:47,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:47,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:47,596 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-03-27 03:54:47,596 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-03-27 03:54:47,597 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:47,597 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:47,597 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,597 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:47,598 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,598 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:47,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,598 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:47,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:47,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:47,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:47,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:47,600 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:47,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:47,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:47,602 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:47,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:47,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:47,604 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:47,604 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:47,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:47,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:47,604 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:47,604 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:47,605 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-03-27 03:54:47,605 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:47,605 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,605 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,605 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:47,605 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,605 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,605 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,605 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:47,605 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:47,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:47,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:47,705 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:47,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:47,707 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38271] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:47,709 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-27 03:54:47,710 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-27 03:54:47,711 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:54:47,711 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679889887711 2023-03-27 03:54:47,712 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-27 03:54:47,713 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-27 03:54:47,713 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-27 03:54:47,713 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] zookeeper.ReadOnlyZKClient(139): Connect 0x101a97d7 to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:54:47,720 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@14f40636, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:54:47,720 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:47,843 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103bdae1942ed404e80a61fdc6dd3679898 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103bdae1942ed404e80a61fdc6dd3679898 2023-03-27 03:54:47,844 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-27 03:54:47,860 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-27 03:54:47,861 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41074, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-27 03:54:47,870 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-27 03:54:47,871 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44626, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-27 03:54:47,887 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/45d43899c39f4f068e6f621d48806a6f first=Optional[r3] last=Optional[r6] 2023-03-27 03:54:47,888 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/45d43899c39f4f068e6f621d48806a6f}] 2023-03-27 03:54:47,894 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/45d43899c39f4f068e6f621d48806a6f for inclusion in 1b540ab2fb7dd087d0e66746ccea8753/family1 2023-03-27 03:54:47,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-03-27 03:54:47,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last= 2023-03-27 03:54:47,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:54:47,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/45d43899c39f4f068e6f621d48806a6f to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__3okktvbd9otn5lt2el4kb2mt2e8j89cm5v0dt69qpd4lksmlcqifahbeja3pi7pr/family1/45d43899c39f4f068e6f621d48806a6f 2023-03-27 03:54:47,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__3okktvbd9otn5lt2el4kb2mt2e8j89cm5v0dt69qpd4lksmlcqifahbeja3pi7pr/family1/45d43899c39f4f068e6f621d48806a6f as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/00f1538b3e9841f39d70e0d5b7704557_SeqId_44_ 2023-03-27 03:54:47,900 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__3okktvbd9otn5lt2el4kb2mt2e8j89cm5v0dt69qpd4lksmlcqifahbeja3pi7pr/family1/45d43899c39f4f068e6f621d48806a6f into 1b540ab2fb7dd087d0e66746ccea8753/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/00f1538b3e9841f39d70e0d5b7704557_SeqId_44_ - updating store file list. 2023-03-27 03:54:47,904 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/00f1538b3e9841f39d70e0d5b7704557_SeqId_44_ into 1b540ab2fb7dd087d0e66746ccea8753/family1 2023-03-27 03:54:47,904 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__3okktvbd9otn5lt2el4kb2mt2e8j89cm5v0dt69qpd4lksmlcqifahbeja3pi7pr/family1/45d43899c39f4f068e6f621d48806a6f into 1b540ab2fb7dd087d0e66746ccea8753/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/00f1538b3e9841f39d70e0d5b7704557_SeqId_44_) 2023-03-27 03:54:47,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__3okktvbd9otn5lt2el4kb2mt2e8j89cm5v0dt69qpd4lksmlcqifahbeja3pi7pr/family1/45d43899c39f4f068e6f621d48806a6f 2023-03-27 03:54:47,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-27 03:54:47,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-27 03:54:47,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(1912): 1b540ab2fb7dd087d0e66746ccea8753/family1 is initiating minor compaction 2023-03-27 03:54:47,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6851136b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:47,909 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1b540ab2fb7dd087d0e66746ccea8753/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:47,909 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/fab57fbeb0eb4ebb9c707e0c187c1032, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/fddf8d78bdb94f0ebcf0e33af2814806, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/489d2ef3acda4336b260abe1ef6414d4, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/879be0ab998b457194fe0279e687ad85, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/07cebda199ec4e349efbd8b96ca98d62, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/2b222396699f41b382f9800b311da70b, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/9030953197314aeabbb90b18ecff4b9b, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/897e4c95464743f89a952b90d38b9608, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/090c94ee37ee41bda6843ba9ed02549a, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/00f1538b3e9841f39d70e0d5b7704557_SeqId_44_] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp, totalSize=50.3 K 2023-03-27 03:54:47,909 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting fab57fbeb0eb4ebb9c707e0c187c1032, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-27 03:54:47,909 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting fddf8d78bdb94f0ebcf0e33af2814806, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-27 03:54:47,909 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 489d2ef3acda4336b260abe1ef6414d4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-27 03:54:47,909 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 879be0ab998b457194fe0279e687ad85, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-27 03:54:47,909 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 07cebda199ec4e349efbd8b96ca98d62, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-27 03:54:47,909 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 2b222396699f41b382f9800b311da70b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-27 03:54:47,909 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 9030953197314aeabbb90b18ecff4b9b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-27 03:54:47,910 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 897e4c95464743f89a952b90d38b9608, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-27 03:54:47,910 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 090c94ee37ee41bda6843ba9ed02549a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-27 03:54:47,910 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 00f1538b3e9841f39d70e0d5b7704557_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-27 03:54:47,985 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x20edfee5 to 127.0.0.1:62246 2023-03-27 03:54:47,985 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:54:47,985 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-27 03:54:47,985 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6d7463d6 to 127.0.0.1:62246 2023-03-27 03:54:47,985 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:54:47,985 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-27 03:54:47,986 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2ab3b343 to 127.0.0.1:62246 2023-03-27 03:54:47,986 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:54:47,986 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-27 03:54:47,986 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x343d14c4 to 127.0.0.1:62246 2023-03-27 03:54:47,986 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:54:47,989 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-27 03:54:48,005 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:48,007 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015122840f1ca9b7a0544e181e01c543e90ff0a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015122840f1ca9b7a0544e181e01c543e90ff0a 2023-03-27 03:54:48,009 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231718504109a1c45d1b43fa65169b99021 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231718504109a1c45d1b43fa65169b99021 2023-03-27 03:54:48,011 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601019a9cf63dd7c44bf6910904f545d91cb9 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601019a9cf63dd7c44bf6910904f545d91cb9 2023-03-27 03:54:48,013 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103324459e088f846938407ad538b58061b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103324459e088f846938407ad538b58061b 2023-03-27 03:54:48,013 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-03-27 03:54:48,013 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-03-27 03:54:48,014 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:48,014 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:48,014 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:48,014 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:48,014 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:48,014 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:48,014 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:48,015 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:48,092 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1b540ab2fb7dd087d0e66746ccea8753#family1#compaction#73 average throughput is 0.07 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:54:48,168 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120574573c4cfb9e49ae8d283951cdb69637 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120574573c4cfb9e49ae8d283951cdb69637 2023-03-27 03:54:48,174 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-27 03:54:48,248 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/71ad65fc55834d898f402232f413f861 first=Optional[r00] last=Optional[r2] 2023-03-27 03:54:48,249 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/71ad65fc55834d898f402232f413f861}] 2023-03-27 03:54:48,251 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/71ad65fc55834d898f402232f413f861 for inclusion in 1b540ab2fb7dd087d0e66746ccea8753/family1 2023-03-27 03:54:48,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-03-27 03:54:48,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last= 2023-03-27 03:54:48,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:54:48,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/71ad65fc55834d898f402232f413f861 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__lbj4odkf8n6ske9u7n01hj9v2d41j8een6ig3fsjgmuq3etrffqc1arhduhsr1p0/family1/71ad65fc55834d898f402232f413f861 2023-03-27 03:54:48,259 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__lbj4odkf8n6ske9u7n01hj9v2d41j8een6ig3fsjgmuq3etrffqc1arhduhsr1p0/family1/71ad65fc55834d898f402232f413f861 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/e0a532309ca545579f1d5b7a8a8b00bc_SeqId_46_ 2023-03-27 03:54:48,260 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__lbj4odkf8n6ske9u7n01hj9v2d41j8een6ig3fsjgmuq3etrffqc1arhduhsr1p0/family1/71ad65fc55834d898f402232f413f861 into 1b540ab2fb7dd087d0e66746ccea8753/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/e0a532309ca545579f1d5b7a8a8b00bc_SeqId_46_ - updating store file list. 2023-03-27 03:54:48,267 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/e0a532309ca545579f1d5b7a8a8b00bc_SeqId_46_ into 1b540ab2fb7dd087d0e66746ccea8753/family1 2023-03-27 03:54:48,267 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__lbj4odkf8n6ske9u7n01hj9v2d41j8een6ig3fsjgmuq3etrffqc1arhduhsr1p0/family1/71ad65fc55834d898f402232f413f861 into 1b540ab2fb7dd087d0e66746ccea8753/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/e0a532309ca545579f1d5b7a8a8b00bc_SeqId_46_) 2023-03-27 03:54:48,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__lbj4odkf8n6ske9u7n01hj9v2d41j8een6ig3fsjgmuq3etrffqc1arhduhsr1p0/family1/71ad65fc55834d898f402232f413f861 2023-03-27 03:54:48,270 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-27 03:54:48,271 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:48,274 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015113033112232b0754428b1e80b6e0e98fa76 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015113033112232b0754428b1e80b6e0e98fa76 2023-03-27 03:54:48,275 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512015f7f06c5482445eebca24cc8dffcd7a4 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512015f7f06c5482445eebca24cc8dffcd7a4 2023-03-27 03:54:48,277 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205efddd440f042426e92add508e370c1ac to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205efddd440f042426e92add508e370c1ac 2023-03-27 03:54:48,277 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-03-27 03:54:48,277 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-03-27 03:54:48,279 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:48,279 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-27 03:54:48,279 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:54:48,280 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-27 03:54:48,282 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 572 msec 2023-03-27 03:54:48,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-27 03:54:48,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:48,523 WARN [DataStreamer for file /user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/0de8c90616fb4e82b4d3a0f5270b3f60] hdfs.DataStreamer(982): Caught exception java.lang.InterruptedException at java.lang.Object.wait(Native Method) at java.lang.Thread.join(Thread.java:1257) at java.lang.Thread.join(Thread.java:1331) at org.apache.hadoop.hdfs.DataStreamer.closeResponder(DataStreamer.java:980) at org.apache.hadoop.hdfs.DataStreamer.endBlock(DataStreamer.java:630) at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:807) 2023-03-27 03:54:48,529 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/0de8c90616fb4e82b4d3a0f5270b3f60 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/0de8c90616fb4e82b4d3a0f5270b3f60 2023-03-27 03:54:48,534 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 1b540ab2fb7dd087d0e66746ccea8753/family1 of 1b540ab2fb7dd087d0e66746ccea8753 into 0de8c90616fb4e82b4d3a0f5270b3f60(size=6.8 K), total size for store is 388.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:54:48,534 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:54:48,534 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1679889287908; duration=0sec 2023-03-27 03:54:48,535 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:48,594 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/fab57fbeb0eb4ebb9c707e0c187c1032, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/fddf8d78bdb94f0ebcf0e33af2814806, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/489d2ef3acda4336b260abe1ef6414d4, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/879be0ab998b457194fe0279e687ad85, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/07cebda199ec4e349efbd8b96ca98d62, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/2b222396699f41b382f9800b311da70b, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/9030953197314aeabbb90b18ecff4b9b, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/897e4c95464743f89a952b90d38b9608, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/090c94ee37ee41bda6843ba9ed02549a, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/00f1538b3e9841f39d70e0d5b7704557_SeqId_44_] to archive 2023-03-27 03:54:48,597 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:48,610 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/fab57fbeb0eb4ebb9c707e0c187c1032 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/fab57fbeb0eb4ebb9c707e0c187c1032 2023-03-27 03:54:48,611 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/fddf8d78bdb94f0ebcf0e33af2814806 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/fddf8d78bdb94f0ebcf0e33af2814806 2023-03-27 03:54:48,614 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/489d2ef3acda4336b260abe1ef6414d4 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/489d2ef3acda4336b260abe1ef6414d4 2023-03-27 03:54:48,616 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/879be0ab998b457194fe0279e687ad85 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/879be0ab998b457194fe0279e687ad85 2023-03-27 03:54:48,617 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/07cebda199ec4e349efbd8b96ca98d62 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/07cebda199ec4e349efbd8b96ca98d62 2023-03-27 03:54:48,618 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/2b222396699f41b382f9800b311da70b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/2b222396699f41b382f9800b311da70b 2023-03-27 03:54:48,620 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/9030953197314aeabbb90b18ecff4b9b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/9030953197314aeabbb90b18ecff4b9b 2023-03-27 03:54:48,621 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/897e4c95464743f89a952b90d38b9608 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/897e4c95464743f89a952b90d38b9608 2023-03-27 03:54:48,622 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/090c94ee37ee41bda6843ba9ed02549a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/090c94ee37ee41bda6843ba9ed02549a 2023-03-27 03:54:48,624 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/00f1538b3e9841f39d70e0d5b7704557_SeqId_44_ to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/00f1538b3e9841f39d70e0d5b7704557_SeqId_44_ 2023-03-27 03:54:48,852 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-27 03:54:48,853 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-03-27 03:54:48,853 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-27 03:54:48,855 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-27 03:54:48,858 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1b540ab2fb7dd087d0e66746ccea8753, REOPEN/MOVE}] 2023-03-27 03:54:48,859 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1b540ab2fb7dd087d0e66746ccea8753, REOPEN/MOVE 2023-03-27 03:54:48,864 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=1b540ab2fb7dd087d0e66746ccea8753, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:48,864 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679889288864"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889288864"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889288864"}]},"ts":"1679889288864"} 2023-03-27 03:54:48,866 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure 1b540ab2fb7dd087d0e66746ccea8753, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:54:49,018 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:49,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1b540ab2fb7dd087d0e66746ccea8753, disabling compactions & flushes 2023-03-27 03:54:49,019 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:49,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:49,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. after waiting 0 ms 2023-03-27 03:54:49,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:49,024 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-03-27 03:54:49,025 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:49,025 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:54:49,025 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 1b540ab2fb7dd087d0e66746ccea8753 to self. 2023-03-27 03:54:49,027 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:49,027 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=1b540ab2fb7dd087d0e66746ccea8753, regionState=CLOSED 2023-03-27 03:54:49,027 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679889289027"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889289027"}]},"ts":"1679889289027"} 2023-03-27 03:54:49,030 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-03-27 03:54:49,030 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure 1b540ab2fb7dd087d0e66746ccea8753, server=jenkins-hbase4.apache.org,39727,1679889218560 in 163 msec 2023-03-27 03:54:49,031 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1b540ab2fb7dd087d0e66746ccea8753, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=true 2023-03-27 03:54:49,181 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=1b540ab2fb7dd087d0e66746ccea8753, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:49,181 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679889289181"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889289181"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889289181"}]},"ts":"1679889289181"} 2023-03-27 03:54:49,183 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure 1b540ab2fb7dd087d0e66746ccea8753, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:54:49,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:49,338 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1b540ab2fb7dd087d0e66746ccea8753, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753.', STARTKEY => '', ENDKEY => ''} 2023-03-27 03:54:49,339 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:49,339 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:49,339 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:49,339 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:49,340 INFO [StoreOpener-1b540ab2fb7dd087d0e66746ccea8753-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:49,341 DEBUG [StoreOpener-1b540ab2fb7dd087d0e66746ccea8753-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1 2023-03-27 03:54:49,341 DEBUG [StoreOpener-1b540ab2fb7dd087d0e66746ccea8753-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1 2023-03-27 03:54:49,342 INFO [StoreOpener-1b540ab2fb7dd087d0e66746ccea8753-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b540ab2fb7dd087d0e66746ccea8753 columnFamilyName family1 2023-03-27 03:54:49,347 DEBUG [StoreOpener-1b540ab2fb7dd087d0e66746ccea8753-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/0de8c90616fb4e82b4d3a0f5270b3f60 2023-03-27 03:54:49,351 DEBUG [StoreOpener-1b540ab2fb7dd087d0e66746ccea8753-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/b4d440078d12497fa79327245c339db4 2023-03-27 03:54:49,355 DEBUG [StoreOpener-1b540ab2fb7dd087d0e66746ccea8753-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/e0a532309ca545579f1d5b7a8a8b00bc_SeqId_46_ 2023-03-27 03:54:49,355 INFO [StoreOpener-1b540ab2fb7dd087d0e66746ccea8753-1] regionserver.HStore(310): Store=1b540ab2fb7dd087d0e66746ccea8753/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:54:49,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:49,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:49,360 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:49,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1b540ab2fb7dd087d0e66746ccea8753; next sequenceid=50; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75153850, jitterRate=0.11987963318824768}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-27 03:54:49,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:54:49,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753., pid=63, masterSystemTime=1679889289335 2023-03-27 03:54:49,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:49,362 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-27 03:54:49,364 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-27 03:54:49,364 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1912): 1b540ab2fb7dd087d0e66746ccea8753/family1 is initiating minor compaction (all files) 2023-03-27 03:54:49,364 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1b540ab2fb7dd087d0e66746ccea8753/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:49,364 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/b4d440078d12497fa79327245c339db4, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/0de8c90616fb4e82b4d3a0f5270b3f60, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/e0a532309ca545579f1d5b7a8a8b00bc_SeqId_46_] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp, totalSize=388.1 K 2023-03-27 03:54:49,364 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=1b540ab2fb7dd087d0e66746ccea8753, regionState=OPEN, openSeqNum=50, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:49,365 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679889289364"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889289364"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889289364"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889289364"}]},"ts":"1679889289364"} 2023-03-27 03:54:49,365 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting b4d440078d12497fa79327245c339db4, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878842162 2023-03-27 03:54:49,365 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:49,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:49,365 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 0de8c90616fb4e82b4d3a0f5270b3f60, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621242162 2023-03-27 03:54:49,366 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting e0a532309ca545579f1d5b7a8a8b00bc_SeqId_46_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448878842162 2023-03-27 03:54:49,369 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-03-27 03:54:49,369 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure 1b540ab2fb7dd087d0e66746ccea8753, server=jenkins-hbase4.apache.org,39727,1679889218560 in 183 msec 2023-03-27 03:54:49,370 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-03-27 03:54:49,370 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1b540ab2fb7dd087d0e66746ccea8753, REOPEN/MOVE in 511 msec 2023-03-27 03:54:49,373 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-03-27 03:54:49,373 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 516 msec 2023-03-27 03:54:49,378 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 950 msec 2023-03-27 03:54:49,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-03-27 03:54:49,380 INFO [Listener at localhost/36131] compactions.TestMobCompactor(278): alter status finished 2023-03-27 03:54:49,381 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38271] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:49,382 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-27 03:54:49,383 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-27 03:54:49,384 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:54:49,384 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679889889384 2023-03-27 03:54:49,386 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-27 03:54:49,387 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-27 03:54:49,387 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-27 03:54:49,387 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] zookeeper.ReadOnlyZKClient(139): Connect 0x31d030e0 to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:54:49,389 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1b540ab2fb7dd087d0e66746ccea8753#family1#compaction#74 average throughput is 16.33 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:54:49,400 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@429b1fa0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:54:49,400 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:49,400 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:49,400 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:49,400 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:49,400 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:49,816 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/.tmp/family1/83b981b62ed0464a92300767f005c4b4 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/83b981b62ed0464a92300767f005c4b4 2023-03-27 03:54:49,820 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 1b540ab2fb7dd087d0e66746ccea8753/family1 of 1b540ab2fb7dd087d0e66746ccea8753 into 83b981b62ed0464a92300767f005c4b4(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:54:49,821 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:54:49,821 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753., storeName=1b540ab2fb7dd087d0e66746ccea8753/family1, priority=13, startTime=1679889289361; duration=0sec 2023-03-27 03:54:49,821 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:49,836 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128821dc9f22b7045db8ce88ed2ef0f34d6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128821dc9f22b7045db8ce88ed2ef0f34d6 2023-03-27 03:54:49,837 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-27 03:54:49,856 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-27 03:54:49,857 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41088, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-27 03:54:49,864 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-27 03:54:49,865 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44642, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-27 03:54:49,878 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/71224d09eb2c4dfdb549e24245321c2c first=Optional[r05] last=Optional[r8] 2023-03-27 03:54:49,878 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=50 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/71224d09eb2c4dfdb549e24245321c2c}] 2023-03-27 03:54:49,883 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/71224d09eb2c4dfdb549e24245321c2c for inclusion in 1b540ab2fb7dd087d0e66746ccea8753/family1 2023-03-27 03:54:49,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-03-27 03:54:49,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last= 2023-03-27 03:54:49,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:54:49,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/71224d09eb2c4dfdb549e24245321c2c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__up9iumtmp6v4o290b4jt03bc6v8grk0fm3746t62nptcqn8ql1n2op4a28p80e8d/family1/71224d09eb2c4dfdb549e24245321c2c 2023-03-27 03:54:49,889 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__up9iumtmp6v4o290b4jt03bc6v8grk0fm3746t62nptcqn8ql1n2op4a28p80e8d/family1/71224d09eb2c4dfdb549e24245321c2c as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/def32eb17cf14ba1aee958a2976e5c9e_SeqId_53_ 2023-03-27 03:54:49,890 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__up9iumtmp6v4o290b4jt03bc6v8grk0fm3746t62nptcqn8ql1n2op4a28p80e8d/family1/71224d09eb2c4dfdb549e24245321c2c into 1b540ab2fb7dd087d0e66746ccea8753/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/def32eb17cf14ba1aee958a2976e5c9e_SeqId_53_ - updating store file list. 2023-03-27 03:54:49,893 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/b4d440078d12497fa79327245c339db4, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/0de8c90616fb4e82b4d3a0f5270b3f60, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/e0a532309ca545579f1d5b7a8a8b00bc_SeqId_46_] to archive 2023-03-27 03:54:49,894 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/def32eb17cf14ba1aee958a2976e5c9e_SeqId_53_ into 1b540ab2fb7dd087d0e66746ccea8753/family1 2023-03-27 03:54:49,894 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__up9iumtmp6v4o290b4jt03bc6v8grk0fm3746t62nptcqn8ql1n2op4a28p80e8d/family1/71224d09eb2c4dfdb549e24245321c2c into 1b540ab2fb7dd087d0e66746ccea8753/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/def32eb17cf14ba1aee958a2976e5c9e_SeqId_53_) 2023-03-27 03:54:49,894 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:49,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__up9iumtmp6v4o290b4jt03bc6v8grk0fm3746t62nptcqn8ql1n2op4a28p80e8d/family1/71224d09eb2c4dfdb549e24245321c2c 2023-03-27 03:54:49,896 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/b4d440078d12497fa79327245c339db4 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/b4d440078d12497fa79327245c339db4 2023-03-27 03:54:49,897 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-27 03:54:49,898 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:49,898 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/0de8c90616fb4e82b4d3a0f5270b3f60 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/0de8c90616fb4e82b4d3a0f5270b3f60 2023-03-27 03:54:49,900 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120b5378f1072454231bbb3ff25931388a6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120b5378f1072454231bbb3ff25931388a6 2023-03-27 03:54:49,901 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/e0a532309ca545579f1d5b7a8a8b00bc_SeqId_46_ to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/e0a532309ca545579f1d5b7a8a8b00bc_SeqId_46_ 2023-03-27 03:54:49,901 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128f2ece1c400c6449697adf1cd5b066492 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128f2ece1c400c6449697adf1cd5b066492 2023-03-27 03:54:49,902 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-03-27 03:54:49,902 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-27 03:54:49,902 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:49,902 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:49,902 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:49,902 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-27 03:54:49,902 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:54:49,904 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-27 03:54:49,905 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 522 msec 2023-03-27 03:54:49,969 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-03-27 03:54:49,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:50,383 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-27 03:54:50,384 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-03-27 03:54:50,384 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-03-27 03:54:50,386 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-03-27 03:54:50,388 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1b540ab2fb7dd087d0e66746ccea8753, REOPEN/MOVE}] 2023-03-27 03:54:50,389 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1b540ab2fb7dd087d0e66746ccea8753, REOPEN/MOVE 2023-03-27 03:54:50,390 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=1b540ab2fb7dd087d0e66746ccea8753, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:50,390 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679889290390"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889290390"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889290390"}]},"ts":"1679889290390"} 2023-03-27 03:54:50,391 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure 1b540ab2fb7dd087d0e66746ccea8753, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:54:50,543 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:50,543 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1b540ab2fb7dd087d0e66746ccea8753, disabling compactions & flushes 2023-03-27 03:54:50,543 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:50,543 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:50,543 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. after waiting 0 ms 2023-03-27 03:54:50,543 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:50,547 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=49 2023-03-27 03:54:50,548 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:50,548 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:54:50,548 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 1b540ab2fb7dd087d0e66746ccea8753 to self. 2023-03-27 03:54:50,549 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:50,550 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=1b540ab2fb7dd087d0e66746ccea8753, regionState=CLOSED 2023-03-27 03:54:50,550 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679889290550"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889290550"}]},"ts":"1679889290550"} 2023-03-27 03:54:50,552 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-03-27 03:54:50,552 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure 1b540ab2fb7dd087d0e66746ccea8753, server=jenkins-hbase4.apache.org,39727,1679889218560 in 160 msec 2023-03-27 03:54:50,553 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1b540ab2fb7dd087d0e66746ccea8753, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=true 2023-03-27 03:54:50,703 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=1b540ab2fb7dd087d0e66746ccea8753, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:50,703 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679889290703"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889290703"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889290703"}]},"ts":"1679889290703"} 2023-03-27 03:54:50,706 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure 1b540ab2fb7dd087d0e66746ccea8753, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:54:50,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:50,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1b540ab2fb7dd087d0e66746ccea8753, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753.', STARTKEY => '', ENDKEY => ''} 2023-03-27 03:54:50,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:50,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:50,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:50,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:50,862 INFO [StoreOpener-1b540ab2fb7dd087d0e66746ccea8753-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:50,863 DEBUG [StoreOpener-1b540ab2fb7dd087d0e66746ccea8753-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1 2023-03-27 03:54:50,863 DEBUG [StoreOpener-1b540ab2fb7dd087d0e66746ccea8753-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1 2023-03-27 03:54:50,863 INFO [StoreOpener-1b540ab2fb7dd087d0e66746ccea8753-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b540ab2fb7dd087d0e66746ccea8753 columnFamilyName family1 2023-03-27 03:54:50,870 DEBUG [StoreOpener-1b540ab2fb7dd087d0e66746ccea8753-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/83b981b62ed0464a92300767f005c4b4 2023-03-27 03:54:50,873 DEBUG [StoreOpener-1b540ab2fb7dd087d0e66746ccea8753-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/family1/def32eb17cf14ba1aee958a2976e5c9e_SeqId_53_ 2023-03-27 03:54:50,873 INFO [StoreOpener-1b540ab2fb7dd087d0e66746ccea8753-1] regionserver.HStore(310): Store=1b540ab2fb7dd087d0e66746ccea8753/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:54:50,874 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:50,875 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:50,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:54:50,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1b540ab2fb7dd087d0e66746ccea8753; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70151324, jitterRate=0.04533618688583374}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-27 03:54:50,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:54:50,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753., pid=69, masterSystemTime=1679889290857 2023-03-27 03:54:50,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:50,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:54:50,882 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=1b540ab2fb7dd087d0e66746ccea8753, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:50,882 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1679889290881"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889290881"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889290881"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889290881"}]},"ts":"1679889290881"} 2023-03-27 03:54:50,884 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-03-27 03:54:50,884 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure 1b540ab2fb7dd087d0e66746ccea8753, server=jenkins-hbase4.apache.org,39727,1679889218560 in 178 msec 2023-03-27 03:54:50,886 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-03-27 03:54:50,886 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=1b540ab2fb7dd087d0e66746ccea8753, REOPEN/MOVE in 496 msec 2023-03-27 03:54:50,887 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-03-27 03:54:50,887 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 500 msec 2023-03-27 03:54:50,889 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 919 msec 2023-03-27 03:54:50,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-03-27 03:54:50,891 INFO [Listener at localhost/36131] compactions.TestMobCompactor(278): alter status finished 2023-03-27 03:54:50,892 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38271] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-03-27 03:54:50,893 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-27 03:54:50,894 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-27 03:54:50,895 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:54:50,895 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679889890895 2023-03-27 03:54:50,895 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-27 03:54:50,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-03-27 03:54:50,896 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-27 03:54:50,897 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] zookeeper.ReadOnlyZKClient(139): Connect 0x13634b6c to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:54:50,901 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1c093208, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:54:50,902 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:50,902 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:50,902 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:50,902 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:50,902 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:50,902 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:50,902 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:50,902 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-03-27 03:54:50,902 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-03-27 03:54:50,902 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:54:50,903 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-03-27 03:54:50,904 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 10 msec 2023-03-27 03:54:50,962 INFO [Listener at localhost/36131] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=245 (was 256), OpenFileDescriptor=621 (was 632), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=348 (was 437), ProcessCount=169 (was 170), AvailableMemoryMB=8780 (was 8807) 2023-03-27 03:54:50,976 INFO [Listener at localhost/36131] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=245, OpenFileDescriptor=621, MaxFileDescriptor=60000, SystemLoadAverage=348, ProcessCount=169, AvailableMemoryMB=8780 2023-03-27 03:54:50,980 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-27 03:54:50,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-03-27 03:54:50,985 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-27 03:54:50,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-03-27 03:54:50,986 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-27 03:54:50,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-27 03:54:50,987 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e 2023-03-27 03:54:50,988 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e empty. 2023-03-27 03:54:50,988 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e 2023-03-27 03:54:50,988 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-27 03:54:51,000 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-03-27 03:54:51,002 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6ae6a8783f210acf27c6043d3e0b573e, NAME => 'testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp 2023-03-27 03:54:51,013 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:51,013 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1604): Closing 6ae6a8783f210acf27c6043d3e0b573e, disabling compactions & flushes 2023-03-27 03:54:51,013 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e. 2023-03-27 03:54:51,013 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e. 2023-03-27 03:54:51,013 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e. after waiting 0 ms 2023-03-27 03:54:51,013 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e. 2023-03-27 03:54:51,013 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e. 2023-03-27 03:54:51,014 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 6ae6a8783f210acf27c6043d3e0b573e: 2023-03-27 03:54:51,016 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-27 03:54:51,017 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1679889291016"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889291016"}]},"ts":"1679889291016"} 2023-03-27 03:54:51,018 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-27 03:54:51,019 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-27 03:54:51,019 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889291019"}]},"ts":"1679889291019"} 2023-03-27 03:54:51,020 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-03-27 03:54:51,028 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=6ae6a8783f210acf27c6043d3e0b573e, ASSIGN}] 2023-03-27 03:54:51,030 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=6ae6a8783f210acf27c6043d3e0b573e, ASSIGN 2023-03-27 03:54:51,031 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=6ae6a8783f210acf27c6043d3e0b573e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=false 2023-03-27 03:54:51,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-27 03:54:51,182 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=6ae6a8783f210acf27c6043d3e0b573e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:51,182 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1679889291182"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889291182"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889291182"}]},"ts":"1679889291182"} 2023-03-27 03:54:51,184 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure 6ae6a8783f210acf27c6043d3e0b573e, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:54:51,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-27 03:54:51,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e. 2023-03-27 03:54:51,340 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6ae6a8783f210acf27c6043d3e0b573e, NAME => 'testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e.', STARTKEY => '', ENDKEY => ''} 2023-03-27 03:54:51,340 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-03-27 03:54:51,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-03-27 03:54:51,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction 6ae6a8783f210acf27c6043d3e0b573e 2023-03-27 03:54:51,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:51,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6ae6a8783f210acf27c6043d3e0b573e 2023-03-27 03:54:51,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6ae6a8783f210acf27c6043d3e0b573e 2023-03-27 03:54:51,344 INFO [StoreOpener-6ae6a8783f210acf27c6043d3e0b573e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 6ae6a8783f210acf27c6043d3e0b573e 2023-03-27 03:54:51,346 DEBUG [StoreOpener-6ae6a8783f210acf27c6043d3e0b573e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1 2023-03-27 03:54:51,346 DEBUG [StoreOpener-6ae6a8783f210acf27c6043d3e0b573e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1 2023-03-27 03:54:51,346 INFO [StoreOpener-6ae6a8783f210acf27c6043d3e0b573e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ae6a8783f210acf27c6043d3e0b573e columnFamilyName f1 2023-03-27 03:54:51,348 INFO [StoreOpener-6ae6a8783f210acf27c6043d3e0b573e-1] regionserver.HStore(310): Store=6ae6a8783f210acf27c6043d3e0b573e/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:54:51,349 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e 2023-03-27 03:54:51,349 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e 2023-03-27 03:54:51,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6ae6a8783f210acf27c6043d3e0b573e 2023-03-27 03:54:51,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:54:51,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6ae6a8783f210acf27c6043d3e0b573e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66885246, jitterRate=-0.003332167863845825}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-27 03:54:51,355 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6ae6a8783f210acf27c6043d3e0b573e: 2023-03-27 03:54:51,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e., pid=73, masterSystemTime=1679889291337 2023-03-27 03:54:51,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e. 2023-03-27 03:54:51,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e. 2023-03-27 03:54:51,358 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=6ae6a8783f210acf27c6043d3e0b573e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:51,358 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1679889291358"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889291358"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889291358"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889291358"}]},"ts":"1679889291358"} 2023-03-27 03:54:51,361 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-03-27 03:54:51,361 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure 6ae6a8783f210acf27c6043d3e0b573e, server=jenkins-hbase4.apache.org,39727,1679889218560 in 176 msec 2023-03-27 03:54:51,363 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-03-27 03:54:51,363 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=6ae6a8783f210acf27c6043d3e0b573e, ASSIGN in 333 msec 2023-03-27 03:54:51,364 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-27 03:54:51,364 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889291364"}]},"ts":"1679889291364"} 2023-03-27 03:54:51,365 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-03-27 03:54:51,368 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-27 03:54:51,369 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 388 msec 2023-03-27 03:54:51,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-03-27 03:54:51,591 INFO [Listener at localhost/36131] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-03-27 03:54:51,591 DEBUG [Listener at localhost/36131] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-03-27 03:54:51,593 INFO [Listener at localhost/36131] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-27 03:54:51,596 INFO [Listener at localhost/36131] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-03-27 03:54:51,596 INFO [Listener at localhost/36131] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-03-27 03:54:51,596 INFO [Listener at localhost/36131] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-03-27 03:54:51,600 INFO [Listener at localhost/36131] regionserver.HRegion(2745): Flushing 6ae6a8783f210acf27c6043d3e0b573e 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-27 03:54:52,018 INFO [Listener at localhost/36131] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230327afff75e238904c3e9b41248ba18613d1 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230327afff75e238904c3e9b41248ba18613d1 2023-03-27 03:54:52,425 INFO [Listener at localhost/36131] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/.tmp/f1/fbf14904018a4053af75294688d624c3 2023-03-27 03:54:52,430 DEBUG [Listener at localhost/36131] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/.tmp/f1/fbf14904018a4053af75294688d624c3 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/fbf14904018a4053af75294688d624c3 2023-03-27 03:54:52,434 INFO [Listener at localhost/36131] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/fbf14904018a4053af75294688d624c3, entries=1, sequenceid=5, filesize=4.9 K 2023-03-27 03:54:52,435 INFO [Listener at localhost/36131] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 6ae6a8783f210acf27c6043d3e0b573e in 835ms, sequenceid=5, compaction requested=false 2023-03-27 03:54:52,435 DEBUG [Listener at localhost/36131] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-03-27 03:54:52,436 DEBUG [Listener at localhost/36131] regionserver.HRegion(2446): Flush status journal for 6ae6a8783f210acf27c6043d3e0b573e: 2023-03-27 03:54:52,438 INFO [Listener at localhost/36131] regionserver.HRegion(2745): Flushing 6ae6a8783f210acf27c6043d3e0b573e 1/1 column families, dataSize=37 B heapSize=368 B 2023-03-27 03:54:52,451 INFO [Listener at localhost/36131] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230327e7ce32a1e0604036a95e257662e3bf7f to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230327e7ce32a1e0604036a95e257662e3bf7f 2023-03-27 03:54:52,469 INFO [Listener at localhost/36131] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/.tmp/f1/f89487a79da24d7a85b3d379d7673180 2023-03-27 03:54:52,474 DEBUG [Listener at localhost/36131] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/.tmp/f1/f89487a79da24d7a85b3d379d7673180 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/f89487a79da24d7a85b3d379d7673180 2023-03-27 03:54:52,477 INFO [Listener at localhost/36131] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/f89487a79da24d7a85b3d379d7673180, entries=1, sequenceid=9, filesize=4.9 K 2023-03-27 03:54:52,478 INFO [Listener at localhost/36131] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 6ae6a8783f210acf27c6043d3e0b573e in 40ms, sequenceid=9, compaction requested=false 2023-03-27 03:54:52,478 DEBUG [Listener at localhost/36131] regionserver.HRegion(2446): Flush status journal for 6ae6a8783f210acf27c6043d3e0b573e: 2023-03-27 03:54:52,480 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38271] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-03-27 03:54:52,481 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-27 03:54:52,482 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-27 03:54:52,483 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:54:52,483 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679889892483 2023-03-27 03:54:52,483 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-27 03:54:52,483 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-03-27 03:54:52,483 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-03-27 03:54:52,484 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] zookeeper.ReadOnlyZKClient(139): Connect 0x6cfd95fd to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:54:52,489 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@39316318, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:54:52,489 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230327 for table='testGetAfterCompaction' and column='f1' 2023-03-27 03:54:52,516 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303275a3332db4dab4afd81ee19f5dbe3bf8b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303275a3332db4dab4afd81ee19f5dbe3bf8b 2023-03-27 03:54:52,517 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230327 table='testGetAfterCompaction' column='f1' 2023-03-27 03:54:52,532 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/f89487a79da24d7a85b3d379d7673180; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292469; access_time=1679889292440; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:52,532 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/fbf14904018a4053af75294688d624c3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292425; access_time=1679889291603; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:52,536 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-27 03:54:52,537 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41102, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-27 03:54:52,543 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-27 03:54:52,545 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44644, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-27 03:54:52,555 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230327/f1/fef667809e7f44069ca5d86e8783a818 first=Optional[r1] last=Optional[r2] 2023-03-27 03:54:52,555 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row with hfile group [{f1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230327/f1/fef667809e7f44069ca5d86e8783a818}] 2023-03-27 03:54:52,559 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230327/f1/fef667809e7f44069ca5d86e8783a818 for inclusion in 6ae6a8783f210acf27c6043d3e0b573e/f1 2023-03-27 03:54:52,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-03-27 03:54:52,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last= 2023-03-27 03:54:52,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for 6ae6a8783f210acf27c6043d3e0b573e: 2023-03-27 03:54:52,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230327/f1/fef667809e7f44069ca5d86e8783a818 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testGetAfterCompaction__3utivkgpq0je1iqr61njq4433iigfhk6r65annkimshkv9no4eqoqirt124002jc/f1/fef667809e7f44069ca5d86e8783a818 2023-03-27 03:54:52,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testGetAfterCompaction__3utivkgpq0je1iqr61njq4433iigfhk6r65annkimshkv9no4eqoqirt124002jc/f1/fef667809e7f44069ca5d86e8783a818 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/707dfc9579044490a70f5f643e877eb9_SeqId_12_ 2023-03-27 03:54:52,568 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testGetAfterCompaction__3utivkgpq0je1iqr61njq4433iigfhk6r65annkimshkv9no4eqoqirt124002jc/f1/fef667809e7f44069ca5d86e8783a818 into 6ae6a8783f210acf27c6043d3e0b573e/f1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/707dfc9579044490a70f5f643e877eb9_SeqId_12_ - updating store file list. 2023-03-27 03:54:52,572 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/707dfc9579044490a70f5f643e877eb9_SeqId_12_ into 6ae6a8783f210acf27c6043d3e0b573e/f1 2023-03-27 03:54:52,572 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testGetAfterCompaction__3utivkgpq0je1iqr61njq4433iigfhk6r65annkimshkv9no4eqoqirt124002jc/f1/fef667809e7f44069ca5d86e8783a818 into 6ae6a8783f210acf27c6043d3e0b573e/f1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/707dfc9579044490a70f5f643e877eb9_SeqId_12_) 2023-03-27 03:54:52,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testGetAfterCompaction__3utivkgpq0je1iqr61njq4433iigfhk6r65annkimshkv9no4eqoqirt124002jc/f1/fef667809e7f44069ca5d86e8783a818 2023-03-27 03:54:52,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(1912): 6ae6a8783f210acf27c6043d3e0b573e/f1 is initiating minor compaction 2023-03-27 03:54:52,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@485b12e; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:52,574 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 6ae6a8783f210acf27c6043d3e0b573e/f1 in testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e. 2023-03-27 03:54:52,574 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/f89487a79da24d7a85b3d379d7673180, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/707dfc9579044490a70f5f643e877eb9_SeqId_12_] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/.tmp, totalSize=10.0 K 2023-03-27 03:54:52,574 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting f89487a79da24d7a85b3d379d7673180, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-27 03:54:52,574 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 707dfc9579044490a70f5f643e877eb9_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-27 03:54:52,576 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230327 table='testGetAfterCompaction' column='f1' 2023-03-27 03:54:52,577 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:52,579 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230327afff75e238904c3e9b41248ba18613d1 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230327afff75e238904c3e9b41248ba18613d1 2023-03-27 03:54:52,580 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230327e7ce32a1e0604036a95e257662e3bf7f to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230327e7ce32a1e0604036a95e257662e3bf7f 2023-03-27 03:54:52,580 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-03-27 03:54:52,580 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230327' 2023-03-27 03:54:52,580 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230327 for table='testGetAfterCompaction' and column='f1' 2023-03-27 03:54:52,582 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-03-27 03:54:52,582 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:54:52,583 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-03-27 03:54:52,583 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/707dfc9579044490a70f5f643e877eb9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679889292512; access_time=1679889292502; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-27 03:54:52,584 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/f89487a79da24d7a85b3d379d7673180; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292469; access_time=1679889292440; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:52,584 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/fbf14904018a4053af75294688d624c3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292425; access_time=1679889291603; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:52,584 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6ae6a8783f210acf27c6043d3e0b573e#f1#compaction#77 average throughput is 0.05 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:54:52,585 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 102 msec 2023-03-27 03:54:52,598 DEBUG [Listener at localhost/36131] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-03-27 03:54:52,605 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41108, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-03-27 03:54:52,605 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39727] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e. 2023-03-27 03:54:52,606 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39727] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-03-27 03:54:52,606 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39727] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-03-27 03:54:52,606 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39727] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-03-27 03:54:52,606 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39727] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e. because compaction request was cancelled 2023-03-27 03:54:52,659 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/707dfc9579044490a70f5f643e877eb9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679889292512; access_time=1679889292502; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-27 03:54:52,659 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/f89487a79da24d7a85b3d379d7673180; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292469; access_time=1679889292440; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:52,659 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/fbf14904018a4053af75294688d624c3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292425; access_time=1679889291603; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:52,710 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/707dfc9579044490a70f5f643e877eb9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679889292512; access_time=1679889292502; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-27 03:54:52,710 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/f89487a79da24d7a85b3d379d7673180; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292469; access_time=1679889292440; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:52,710 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/fbf14904018a4053af75294688d624c3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292425; access_time=1679889291603; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:52,761 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/707dfc9579044490a70f5f643e877eb9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679889292512; access_time=1679889292502; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-27 03:54:52,761 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/f89487a79da24d7a85b3d379d7673180; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292469; access_time=1679889292440; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:52,761 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/fbf14904018a4053af75294688d624c3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292425; access_time=1679889291603; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:52,812 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/707dfc9579044490a70f5f643e877eb9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679889292512; access_time=1679889292502; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-27 03:54:52,812 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/f89487a79da24d7a85b3d379d7673180; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292469; access_time=1679889292440; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:52,812 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/fbf14904018a4053af75294688d624c3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292425; access_time=1679889291603; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:52,863 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/707dfc9579044490a70f5f643e877eb9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679889292512; access_time=1679889292502; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-27 03:54:52,863 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/f89487a79da24d7a85b3d379d7673180; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292469; access_time=1679889292440; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:52,863 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/fbf14904018a4053af75294688d624c3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292425; access_time=1679889291603; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:52,914 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/707dfc9579044490a70f5f643e877eb9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679889292512; access_time=1679889292502; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-27 03:54:52,914 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/f89487a79da24d7a85b3d379d7673180; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292469; access_time=1679889292440; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:52,915 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/fbf14904018a4053af75294688d624c3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292425; access_time=1679889291603; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:52,965 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/707dfc9579044490a70f5f643e877eb9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679889292512; access_time=1679889292502; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-27 03:54:52,965 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/f89487a79da24d7a85b3d379d7673180; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292469; access_time=1679889292440; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:52,965 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/fbf14904018a4053af75294688d624c3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292425; access_time=1679889291603; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:53,013 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/.tmp/f1/c16a4a631a3744a69db086e191bb3cbd as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/c16a4a631a3744a69db086e191bb3cbd 2023-03-27 03:54:53,016 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/707dfc9579044490a70f5f643e877eb9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679889292512; access_time=1679889292502; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-27 03:54:53,016 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/c16a4a631a3744a69db086e191bb3cbd; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1679889293007; access_time=1679889292579; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:53,016 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/f89487a79da24d7a85b3d379d7673180; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292469; access_time=1679889292440; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:53,016 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/fbf14904018a4053af75294688d624c3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292425; access_time=1679889291603; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:53,019 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in 6ae6a8783f210acf27c6043d3e0b573e/f1 of 6ae6a8783f210acf27c6043d3e0b573e into c16a4a631a3744a69db086e191bb3cbd(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:54:53,019 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6ae6a8783f210acf27c6043d3e0b573e: 2023-03-27 03:54:53,019 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1679889292573; duration=0sec 2023-03-27 03:54:53,019 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:54:53,067 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/707dfc9579044490a70f5f643e877eb9_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1679889292512; access_time=1679889292502; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-03-27 03:54:53,067 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/c16a4a631a3744a69db086e191bb3cbd; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1679889293007; access_time=1679889292579; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:53,067 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/f89487a79da24d7a85b3d379d7673180; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292469; access_time=1679889292440; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:53,067 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/fbf14904018a4053af75294688d624c3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292425; access_time=1679889291603; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:53,093 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/f89487a79da24d7a85b3d379d7673180, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/707dfc9579044490a70f5f643e877eb9_SeqId_12_] to archive 2023-03-27 03:54:53,093 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:54:53,096 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/f89487a79da24d7a85b3d379d7673180 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/f89487a79da24d7a85b3d379d7673180 2023-03-27 03:54:53,097 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/707dfc9579044490a70f5f643e877eb9_SeqId_12_ to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/707dfc9579044490a70f5f643e877eb9_SeqId_12_ 2023-03-27 03:54:53,118 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/c16a4a631a3744a69db086e191bb3cbd; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1679889293007; access_time=1679889292579; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:53,118 INFO [Listener at localhost/36131] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/fbf14904018a4053af75294688d624c3; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1679889292425; access_time=1679889291603; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-03-27 03:54:53,251 INFO [Listener at localhost/36131] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-03-27 03:54:53,257 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-03-27 03:54:53,263 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-03-27 03:54:53,266 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889293266"}]},"ts":"1679889293266"} 2023-03-27 03:54:53,268 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-03-27 03:54:53,270 INFO [PEWorker-4] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-03-27 03:54:53,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-27 03:54:53,272 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=6ae6a8783f210acf27c6043d3e0b573e, UNASSIGN}] 2023-03-27 03:54:53,273 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=6ae6a8783f210acf27c6043d3e0b573e, UNASSIGN 2023-03-27 03:54:53,274 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=6ae6a8783f210acf27c6043d3e0b573e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:53,274 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1679889293274"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889293274"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889293274"}]},"ts":"1679889293274"} 2023-03-27 03:54:53,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure 6ae6a8783f210acf27c6043d3e0b573e, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:54:53,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-27 03:54:53,427 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 6ae6a8783f210acf27c6043d3e0b573e 2023-03-27 03:54:53,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6ae6a8783f210acf27c6043d3e0b573e, disabling compactions & flushes 2023-03-27 03:54:53,427 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e. 2023-03-27 03:54:53,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e. 2023-03-27 03:54:53,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e. after waiting 0 ms 2023-03-27 03:54:53,427 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e. 2023-03-27 03:54:53,431 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-27 03:54:53,432 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-03-27 03:54:53,432 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e. 2023-03-27 03:54:53,432 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6ae6a8783f210acf27c6043d3e0b573e: 2023-03-27 03:54:53,434 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 6ae6a8783f210acf27c6043d3e0b573e 2023-03-27 03:54:53,434 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=6ae6a8783f210acf27c6043d3e0b573e, regionState=CLOSED 2023-03-27 03:54:53,434 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1679889293434"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889293434"}]},"ts":"1679889293434"} 2023-03-27 03:54:53,437 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-03-27 03:54:53,437 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure 6ae6a8783f210acf27c6043d3e0b573e, server=jenkins-hbase4.apache.org,39727,1679889218560 in 161 msec 2023-03-27 03:54:53,438 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-03-27 03:54:53,439 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=6ae6a8783f210acf27c6043d3e0b573e, UNASSIGN in 165 msec 2023-03-27 03:54:53,439 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889293439"}]},"ts":"1679889293439"} 2023-03-27 03:54:53,440 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-03-27 03:54:53,444 INFO [PEWorker-4] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-03-27 03:54:53,446 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 187 msec 2023-03-27 03:54:53,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-03-27 03:54:53,581 INFO [Listener at localhost/36131] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-03-27 03:54:53,587 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-03-27 03:54:53,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-03-27 03:54:53,598 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-27 03:54:53,600 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-27 03:54:53,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-27 03:54:53,603 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e 2023-03-27 03:54:53,630 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1, FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/recovered.edits] 2023-03-27 03:54:53,635 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/c16a4a631a3744a69db086e191bb3cbd to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/c16a4a631a3744a69db086e191bb3cbd 2023-03-27 03:54:53,638 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/fbf14904018a4053af75294688d624c3 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/f1/fbf14904018a4053af75294688d624c3 2023-03-27 03:54:53,642 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/recovered.edits/15.seqid to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e/recovered.edits/15.seqid 2023-03-27 03:54:53,645 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testGetAfterCompaction/6ae6a8783f210acf27c6043d3e0b573e 2023-03-27 03:54:53,645 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-03-27 03:54:53,645 DEBUG [PEWorker-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-27 03:54:53,646 DEBUG [PEWorker-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-03-27 03:54:53,650 DEBUG [PEWorker-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303275a3332db4dab4afd81ee19f5dbe3bf8b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202303275a3332db4dab4afd81ee19f5dbe3bf8b 2023-03-27 03:54:53,652 DEBUG [PEWorker-1] backup.HFileArchiver(596): Deleted hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-03-27 03:54:53,664 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-27 03:54:53,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-27 03:54:53,711 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-03-27 03:54:53,724 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-03-27 03:54:53,726 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-27 03:54:53,726 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-03-27 03:54:53,726 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1679889293726"}]},"ts":"9223372036854775807"} 2023-03-27 03:54:53,744 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-03-27 03:54:53,744 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 6ae6a8783f210acf27c6043d3e0b573e, NAME => 'testGetAfterCompaction,,1679889290980.6ae6a8783f210acf27c6043d3e0b573e.', STARTKEY => '', ENDKEY => ''}] 2023-03-27 03:54:53,744 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-03-27 03:54:53,744 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1679889293744"}]},"ts":"9223372036854775807"} 2023-03-27 03:54:53,750 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-03-27 03:54:53,752 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-03-27 03:54:53,753 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 164 msec 2023-03-27 03:54:53,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-03-27 03:54:53,904 INFO [Listener at localhost/36131] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-03-27 03:54:53,921 INFO [Listener at localhost/36131] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=253 (was 245) - Thread LEAK? -, OpenFileDescriptor=630 (was 621) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=348 (was 348), ProcessCount=169 (was 169), AvailableMemoryMB=8784 (was 8780) - AvailableMemoryMB LEAK? - 2023-03-27 03:54:53,939 INFO [Listener at localhost/36131] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=253, OpenFileDescriptor=630, MaxFileDescriptor=60000, SystemLoadAverage=348, ProcessCount=169, AvailableMemoryMB=8784 2023-03-27 03:54:53,942 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-27 03:54:53,943 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-03-27 03:54:53,945 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-03-27 03:54:53,946 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-03-27 03:54:53,946 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-27 03:54:53,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-27 03:54:53,950 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:54:53,952 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2 empty. 2023-03-27 03:54:53,952 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:54:53,954 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:54:53,960 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64 empty. 2023-03-27 03:54:53,960 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:54:53,960 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:54:53,961 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2 empty. 2023-03-27 03:54:53,961 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:54:53,961 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-03-27 03:54:54,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-27 03:54:54,250 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-27 03:54:54,382 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-27 03:54:54,384 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7675): creating {ENCODED => a96b2587b7874c72fc83ce1239d003a2, NAME => 'testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp 2023-03-27 03:54:54,385 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7675): creating {ENCODED => faf5e912b9b1c732d5f1903dc1242b64, NAME => 'testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp 2023-03-27 03:54:54,385 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7675): creating {ENCODED => 08cac3032353cd1d586bb4d48f4a89b2, NAME => 'testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp 2023-03-27 03:54:54,551 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-27 03:54:54,562 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-27 03:54:54,806 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:54,806 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1604): Closing 08cac3032353cd1d586bb4d48f4a89b2, disabling compactions & flushes 2023-03-27 03:54:54,806 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:54:54,807 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:54:54,807 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. after waiting 0 ms 2023-03-27 03:54:54,807 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:54:54,807 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:54:54,807 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1558): Region close journal for 08cac3032353cd1d586bb4d48f4a89b2: 2023-03-27 03:54:54,818 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:54,818 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1604): Closing faf5e912b9b1c732d5f1903dc1242b64, disabling compactions & flushes 2023-03-27 03:54:54,818 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:54:54,818 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:54:54,818 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. after waiting 0 ms 2023-03-27 03:54:54,818 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:54:54,818 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:54:54,818 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1558): Region close journal for faf5e912b9b1c732d5f1903dc1242b64: 2023-03-27 03:54:54,819 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:54,819 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1604): Closing a96b2587b7874c72fc83ce1239d003a2, disabling compactions & flushes 2023-03-27 03:54:54,819 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:54:54,819 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:54:54,819 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. after waiting 0 ms 2023-03-27 03:54:54,819 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:54:54,819 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:54:54,819 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1558): Region close journal for a96b2587b7874c72fc83ce1239d003a2: 2023-03-27 03:54:54,822 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-03-27 03:54:54,823 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679889294823"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889294823"}]},"ts":"1679889294823"} 2023-03-27 03:54:54,823 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679889294823"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889294823"}]},"ts":"1679889294823"} 2023-03-27 03:54:54,823 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679889294823"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889294823"}]},"ts":"1679889294823"} 2023-03-27 03:54:54,826 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-27 03:54:54,827 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-27 03:54:54,827 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889294827"}]},"ts":"1679889294827"} 2023-03-27 03:54:54,828 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-27 03:54:54,833 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=08cac3032353cd1d586bb4d48f4a89b2, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=faf5e912b9b1c732d5f1903dc1242b64, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a96b2587b7874c72fc83ce1239d003a2, ASSIGN}] 2023-03-27 03:54:54,836 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a96b2587b7874c72fc83ce1239d003a2, ASSIGN 2023-03-27 03:54:54,836 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=faf5e912b9b1c732d5f1903dc1242b64, ASSIGN 2023-03-27 03:54:54,836 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=08cac3032353cd1d586bb4d48f4a89b2, ASSIGN 2023-03-27 03:54:54,837 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a96b2587b7874c72fc83ce1239d003a2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=false 2023-03-27 03:54:54,837 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=faf5e912b9b1c732d5f1903dc1242b64, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=false 2023-03-27 03:54:54,837 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=08cac3032353cd1d586bb4d48f4a89b2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=false 2023-03-27 03:54:54,989 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=a96b2587b7874c72fc83ce1239d003a2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:54,989 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=08cac3032353cd1d586bb4d48f4a89b2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:54,989 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=faf5e912b9b1c732d5f1903dc1242b64, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:54,989 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679889294989"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889294989"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889294989"}]},"ts":"1679889294989"} 2023-03-27 03:54:54,989 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679889294989"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889294989"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889294989"}]},"ts":"1679889294989"} 2023-03-27 03:54:54,990 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679889294989"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889294989"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889294989"}]},"ts":"1679889294989"} 2023-03-27 03:54:54,991 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=82, state=RUNNABLE; OpenRegionProcedure a96b2587b7874c72fc83ce1239d003a2, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:54:54,992 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=80, state=RUNNABLE; OpenRegionProcedure 08cac3032353cd1d586bb4d48f4a89b2, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:54:54,994 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=81, state=RUNNABLE; OpenRegionProcedure faf5e912b9b1c732d5f1903dc1242b64, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:54:55,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-27 03:54:55,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:54:55,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a96b2587b7874c72fc83ce1239d003a2, NAME => 'testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2.', STARTKEY => '2', ENDKEY => ''} 2023-03-27 03:54:55,149 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:54:55,149 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:55,149 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:54:55,149 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:54:55,155 INFO [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:54:55,157 DEBUG [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1 2023-03-27 03:54:55,157 DEBUG [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1 2023-03-27 03:54:55,157 INFO [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a96b2587b7874c72fc83ce1239d003a2 columnFamilyName family1 2023-03-27 03:54:55,159 INFO [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] regionserver.HStore(310): Store=a96b2587b7874c72fc83ce1239d003a2/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:54:55,159 INFO [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:54:55,173 DEBUG [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2 2023-03-27 03:54:55,173 DEBUG [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2 2023-03-27 03:54:55,174 INFO [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a96b2587b7874c72fc83ce1239d003a2 columnFamilyName family2 2023-03-27 03:54:55,175 INFO [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] regionserver.HStore(310): Store=a96b2587b7874c72fc83ce1239d003a2/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:54:55,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:54:55,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:54:55,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-27 03:54:55,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:54:55,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:54:55,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a96b2587b7874c72fc83ce1239d003a2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66617470, jitterRate=-0.007322341203689575}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-27 03:54:55,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a96b2587b7874c72fc83ce1239d003a2: 2023-03-27 03:54:55,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2., pid=83, masterSystemTime=1679889295145 2023-03-27 03:54:55,186 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=a96b2587b7874c72fc83ce1239d003a2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:55,187 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679889295186"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889295186"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889295186"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889295186"}]},"ts":"1679889295186"} 2023-03-27 03:54:55,187 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:54:55,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:54:55,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:54:55,187 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => faf5e912b9b1c732d5f1903dc1242b64, NAME => 'testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64.', STARTKEY => '1', ENDKEY => '2'} 2023-03-27 03:54:55,187 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:54:55,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:55,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:54:55,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:54:55,191 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=82 2023-03-27 03:54:55,191 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=82, state=SUCCESS; OpenRegionProcedure a96b2587b7874c72fc83ce1239d003a2, server=jenkins-hbase4.apache.org,39727,1679889218560 in 198 msec 2023-03-27 03:54:55,192 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a96b2587b7874c72fc83ce1239d003a2, ASSIGN in 358 msec 2023-03-27 03:54:55,204 INFO [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:54:55,207 DEBUG [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1 2023-03-27 03:54:55,207 DEBUG [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1 2023-03-27 03:54:55,208 INFO [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region faf5e912b9b1c732d5f1903dc1242b64 columnFamilyName family1 2023-03-27 03:54:55,209 INFO [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] regionserver.HStore(310): Store=faf5e912b9b1c732d5f1903dc1242b64/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:54:55,210 INFO [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:54:55,212 DEBUG [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2 2023-03-27 03:54:55,212 DEBUG [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2 2023-03-27 03:54:55,213 INFO [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region faf5e912b9b1c732d5f1903dc1242b64 columnFamilyName family2 2023-03-27 03:54:55,214 INFO [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] regionserver.HStore(310): Store=faf5e912b9b1c732d5f1903dc1242b64/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:54:55,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:54:55,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:54:55,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-27 03:54:55,219 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:54:55,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:54:55,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened faf5e912b9b1c732d5f1903dc1242b64; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72801320, jitterRate=0.08482420444488525}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-27 03:54:55,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for faf5e912b9b1c732d5f1903dc1242b64: 2023-03-27 03:54:55,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64., pid=85, masterSystemTime=1679889295145 2023-03-27 03:54:55,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:54:55,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:54:55,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:54:55,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 08cac3032353cd1d586bb4d48f4a89b2, NAME => 'testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2.', STARTKEY => '', ENDKEY => '1'} 2023-03-27 03:54:55,226 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=faf5e912b9b1c732d5f1903dc1242b64, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:55,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:54:55,226 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679889295226"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889295226"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889295226"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889295226"}]},"ts":"1679889295226"} 2023-03-27 03:54:55,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:54:55,227 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:54:55,227 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:54:55,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=81 2023-03-27 03:54:55,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=81, state=SUCCESS; OpenRegionProcedure faf5e912b9b1c732d5f1903dc1242b64, server=jenkins-hbase4.apache.org,39727,1679889218560 in 235 msec 2023-03-27 03:54:55,230 INFO [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:54:55,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=faf5e912b9b1c732d5f1903dc1242b64, ASSIGN in 399 msec 2023-03-27 03:54:55,232 DEBUG [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1 2023-03-27 03:54:55,232 DEBUG [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1 2023-03-27 03:54:55,233 INFO [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 08cac3032353cd1d586bb4d48f4a89b2 columnFamilyName family1 2023-03-27 03:54:55,234 INFO [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] regionserver.HStore(310): Store=08cac3032353cd1d586bb4d48f4a89b2/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:54:55,234 INFO [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:54:55,236 DEBUG [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2 2023-03-27 03:54:55,236 DEBUG [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2 2023-03-27 03:54:55,237 INFO [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 08cac3032353cd1d586bb4d48f4a89b2 columnFamilyName family2 2023-03-27 03:54:55,238 INFO [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] regionserver.HStore(310): Store=08cac3032353cd1d586bb4d48f4a89b2/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:54:55,239 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:54:55,239 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:54:55,244 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-27 03:54:55,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:54:55,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:54:55,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 08cac3032353cd1d586bb4d48f4a89b2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70895181, jitterRate=0.056420519948005676}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-27 03:54:55,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 08cac3032353cd1d586bb4d48f4a89b2: 2023-03-27 03:54:55,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2., pid=84, masterSystemTime=1679889295145 2023-03-27 03:54:55,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:54:55,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:54:55,252 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=08cac3032353cd1d586bb4d48f4a89b2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:55,252 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679889295252"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889295252"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889295252"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889295252"}]},"ts":"1679889295252"} 2023-03-27 03:54:55,255 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=80 2023-03-27 03:54:55,255 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=80, state=SUCCESS; OpenRegionProcedure 08cac3032353cd1d586bb4d48f4a89b2, server=jenkins-hbase4.apache.org,39727,1679889218560 in 262 msec 2023-03-27 03:54:55,257 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-03-27 03:54:55,257 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=08cac3032353cd1d586bb4d48f4a89b2, ASSIGN in 425 msec 2023-03-27 03:54:55,258 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-27 03:54:55,258 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889295258"}]},"ts":"1679889295258"} 2023-03-27 03:54:55,259 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-27 03:54:55,262 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-03-27 03:54:55,263 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.3200 sec 2023-03-27 03:54:56,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-03-27 03:54:56,054 INFO [Listener at localhost/36131] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-03-27 03:54:56,064 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:54:56,065 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:54:56,066 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:54:56,070 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:56,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-27 03:54:56,073 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-27 03:54:56,073 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:56,073 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-27 03:54:56,073 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-27 03:54:56,073 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:54:56,073 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:54:56,075 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:56,075 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:56,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:56,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:56,075 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:56,075 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:56,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:54:56,076 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:54:56,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:56,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:54:56,076 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-27 03:54:56,076 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-27 03:54:56,076 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-27 03:54:56,077 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:56,077 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-27 03:54:56,080 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:54:56,080 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. started... 2023-03-27 03:54:56,080 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:54:56,080 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. started... 2023-03-27 03:54:56,081 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing faf5e912b9b1c732d5f1903dc1242b64 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-27 03:54:56,081 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:54:56,081 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-27 03:54:56,082 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:56,081 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 08cac3032353cd1d586bb4d48f4a89b2 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-27 03:54:56,082 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. started... 2023-03-27 03:54:56,082 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing a96b2587b7874c72fc83ce1239d003a2 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-27 03:54:56,126 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230327c743126e94f14fd49aa60700919794b4 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230327c743126e94f14fd49aa60700919794b4 2023-03-27 03:54:56,129 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230327a6b65f3b1d404ecc864089d09033c520 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230327a6b65f3b1d404ecc864089d09033c520 2023-03-27 03:54:56,526 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303272e2c3641420440f5aac83f42c82b6a1d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303272e2c3641420440f5aac83f42c82b6a1d 2023-03-27 03:54:56,535 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family1/72ab7fd6b53b42a1ac7eed9e4568659d 2023-03-27 03:54:56,541 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family1/2a10d8329f234324aec49aae55e72329 2023-03-27 03:54:56,932 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family1/bdd2fd9286284264bb4230a42ecd2604 2023-03-27 03:54:56,947 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303275668c83e5a6b43feb6e5d72e713bfa77 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303275668c83e5a6b43feb6e5d72e713bfa77 2023-03-27 03:54:56,952 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303271730ef134362450aac84bafaa4706420 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303271730ef134362450aac84bafaa4706420 2023-03-27 03:54:56,961 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303272376dedadec0473e89d88d6fe66d697a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303272376dedadec0473e89d88d6fe66d697a 2023-03-27 03:54:57,353 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family2/ba625f70d9564e14ac3e61abdbd37e22 2023-03-27 03:54:57,357 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family2/9ca06fff42104415a8c39a7223a455af 2023-03-27 03:54:57,358 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family1/bdd2fd9286284264bb4230a42ecd2604 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/bdd2fd9286284264bb4230a42ecd2604 2023-03-27 03:54:57,363 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/bdd2fd9286284264bb4230a42ecd2604, entries=4, sequenceid=5, filesize=5.4 K 2023-03-27 03:54:57,363 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family1/72ab7fd6b53b42a1ac7eed9e4568659d as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/72ab7fd6b53b42a1ac7eed9e4568659d 2023-03-27 03:54:57,364 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family2/ba625f70d9564e14ac3e61abdbd37e22 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/ba625f70d9564e14ac3e61abdbd37e22 2023-03-27 03:54:57,368 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/72ab7fd6b53b42a1ac7eed9e4568659d, entries=4, sequenceid=5, filesize=5.4 K 2023-03-27 03:54:57,368 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/ba625f70d9564e14ac3e61abdbd37e22, entries=2, sequenceid=5, filesize=5.1 K 2023-03-27 03:54:57,368 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family2/9ca06fff42104415a8c39a7223a455af as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/9ca06fff42104415a8c39a7223a455af 2023-03-27 03:54:57,369 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 08cac3032353cd1d586bb4d48f4a89b2 in 1288ms, sequenceid=5, compaction requested=false 2023-03-27 03:54:57,369 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-03-27 03:54:57,370 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 08cac3032353cd1d586bb4d48f4a89b2: 2023-03-27 03:54:57,370 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:54:57,370 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-27 03:54:57,372 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/9ca06fff42104415a8c39a7223a455af, entries=2, sequenceid=5, filesize=5.1 K 2023-03-27 03:54:57,373 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for a96b2587b7874c72fc83ce1239d003a2 in 1291ms, sequenceid=5, compaction requested=false 2023-03-27 03:54:57,374 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for a96b2587b7874c72fc83ce1239d003a2: 2023-03-27 03:54:57,374 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:54:57,374 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-27 03:54:57,375 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family2/0af7c02db50945349a2e0c8532efb61d 2023-03-27 03:54:57,379 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family1/2a10d8329f234324aec49aae55e72329 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/2a10d8329f234324aec49aae55e72329 2023-03-27 03:54:57,383 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/2a10d8329f234324aec49aae55e72329, entries=4, sequenceid=5, filesize=5.4 K 2023-03-27 03:54:57,384 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family2/0af7c02db50945349a2e0c8532efb61d as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/0af7c02db50945349a2e0c8532efb61d 2023-03-27 03:54:57,389 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/0af7c02db50945349a2e0c8532efb61d, entries=2, sequenceid=5, filesize=5.1 K 2023-03-27 03:54:57,390 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for faf5e912b9b1c732d5f1903dc1242b64 in 1309ms, sequenceid=5, compaction requested=false 2023-03-27 03:54:57,390 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for faf5e912b9b1c732d5f1903dc1242b64: 2023-03-27 03:54:57,390 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:54:57,390 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-27 03:54:57,390 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-27 03:54:57,390 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:57,390 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-27 03:54:57,390 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-27 03:54:57,393 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:57,393 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:54:57,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:57,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:57,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:57,393 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:54:57,393 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:57,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:57,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:57,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:54:57,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:57,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:57,395 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-27 03:54:57,395 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@135f5262[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:57,395 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-27 03:54:57,395 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:54:57,396 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:54:57,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:54:57,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:54:57,396 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:57,396 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:57,396 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-27 03:54:57,397 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-27 03:54:57,397 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:57,398 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:57,398 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-27 03:54:57,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:57,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:57,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:57,398 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:57,398 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-27 03:54:57,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:57,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:57,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:54:57,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:57,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:57,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:54:57,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:57,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:57,401 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-27 03:54:57,401 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-27 03:54:57,402 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:57,402 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:57,402 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-27 03:54:57,402 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:57,403 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:54:57,403 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:54:57,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:54:57,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:57,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:57,403 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:54:57,403 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:57,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:54:57,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:57,404 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:57,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:57,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:54:57,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:57,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:54:57,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:57,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:54:57,406 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:57,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:57,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:57,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:54:57,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:57,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:57,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:57,410 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:57,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:57,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:57,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:57,410 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-27 03:54:57,410 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:57,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:54:57,410 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-27 03:54:57,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:54:57,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:57,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:54:57,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:54:57,410 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:54:57,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:57,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:57,411 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:57,411 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:57,511 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:57,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:57,513 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:54:57,513 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:54:57,513 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:54:57,516 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:57,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:57,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-27 03:54:57,521 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-27 03:54:57,521 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:57,521 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-27 03:54:57,521 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-27 03:54:57,521 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:54:57,521 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:54:57,522 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:57,522 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:57,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:57,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:57,523 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:57,523 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:57,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:54:57,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:54:57,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:57,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:54:57,524 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-27 03:54:57,524 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-27 03:54:57,524 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-27 03:54:57,524 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-27 03:54:57,524 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:57,524 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-27 03:54:57,524 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:54:57,525 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. started... 2023-03-27 03:54:57,525 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:54:57,525 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. started... 2023-03-27 03:54:57,525 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing faf5e912b9b1c732d5f1903dc1242b64 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-27 03:54:57,525 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-27 03:54:57,525 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 08cac3032353cd1d586bb4d48f4a89b2 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-27 03:54:57,525 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:57,525 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:54:57,525 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. started... 2023-03-27 03:54:57,525 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing a96b2587b7874c72fc83ce1239d003a2 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-27 03:54:57,951 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032771c38408a9c64734b2b9a0fc42690dc7 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032771c38408a9c64734b2b9a0fc42690dc7 2023-03-27 03:54:57,953 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032762936666bd29429093aab0b1a3958593 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023032762936666bd29429093aab0b1a3958593 2023-03-27 03:54:57,960 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family1/64b1111da52f413e9f2a26d0594a1069 2023-03-27 03:54:57,961 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family1/a02d2e3e10344b54bba8a663ef52c568 2023-03-27 03:54:57,963 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230327059e3f0cf9874fe597ffaaa4cf3e8f2c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230327059e3f0cf9874fe597ffaaa4cf3e8f2c 2023-03-27 03:54:58,009 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230327fa6e6618f562408ca885c0890360c478 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230327fa6e6618f562408ca885c0890360c478 2023-03-27 03:54:58,014 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family2/4c5d382fe46d43cdb4bbba094ca5be83 2023-03-27 03:54:58,018 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family1/64b1111da52f413e9f2a26d0594a1069 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/64b1111da52f413e9f2a26d0594a1069 2023-03-27 03:54:58,022 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/64b1111da52f413e9f2a26d0594a1069, entries=4, sequenceid=9, filesize=5.4 K 2023-03-27 03:54:58,023 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family2/4c5d382fe46d43cdb4bbba094ca5be83 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/4c5d382fe46d43cdb4bbba094ca5be83 2023-03-27 03:54:58,026 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/4c5d382fe46d43cdb4bbba094ca5be83, entries=2, sequenceid=9, filesize=5.1 K 2023-03-27 03:54:58,027 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 08cac3032353cd1d586bb4d48f4a89b2 in 502ms, sequenceid=9, compaction requested=false 2023-03-27 03:54:58,027 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 08cac3032353cd1d586bb4d48f4a89b2: 2023-03-27 03:54:58,027 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:54:58,027 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-27 03:54:58,370 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family1/5ae08f2ca4ae401f9e6acd9dfb0f908a 2023-03-27 03:54:58,401 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230327ece6da9680f44eda9d86b37189e09673 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230327ece6da9680f44eda9d86b37189e09673 2023-03-27 03:54:58,787 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032764acdbba253f4ebc945ddf3c8b1cdd81 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023032764acdbba253f4ebc945ddf3c8b1cdd81 2023-03-27 03:54:58,807 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family2/0ea65611c6ce4b80a7fa1a7b13e13710 2023-03-27 03:54:58,812 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family1/a02d2e3e10344b54bba8a663ef52c568 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/a02d2e3e10344b54bba8a663ef52c568 2023-03-27 03:54:58,816 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/a02d2e3e10344b54bba8a663ef52c568, entries=4, sequenceid=9, filesize=5.4 K 2023-03-27 03:54:58,817 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family2/0ea65611c6ce4b80a7fa1a7b13e13710 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/0ea65611c6ce4b80a7fa1a7b13e13710 2023-03-27 03:54:58,821 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/0ea65611c6ce4b80a7fa1a7b13e13710, entries=2, sequenceid=9, filesize=5.1 K 2023-03-27 03:54:58,822 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for a96b2587b7874c72fc83ce1239d003a2 in 1296ms, sequenceid=9, compaction requested=false 2023-03-27 03:54:58,822 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for a96b2587b7874c72fc83ce1239d003a2: 2023-03-27 03:54:58,822 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:54:58,822 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-27 03:54:59,193 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family2/7f0e82322b054d29ae0a7124a90065be 2023-03-27 03:54:59,204 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family1/5ae08f2ca4ae401f9e6acd9dfb0f908a as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/5ae08f2ca4ae401f9e6acd9dfb0f908a 2023-03-27 03:54:59,210 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/5ae08f2ca4ae401f9e6acd9dfb0f908a, entries=4, sequenceid=9, filesize=5.4 K 2023-03-27 03:54:59,210 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family2/7f0e82322b054d29ae0a7124a90065be as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/7f0e82322b054d29ae0a7124a90065be 2023-03-27 03:54:59,215 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/7f0e82322b054d29ae0a7124a90065be, entries=2, sequenceid=9, filesize=5.1 K 2023-03-27 03:54:59,216 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for faf5e912b9b1c732d5f1903dc1242b64 in 1691ms, sequenceid=9, compaction requested=false 2023-03-27 03:54:59,216 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for faf5e912b9b1c732d5f1903dc1242b64: 2023-03-27 03:54:59,216 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:54:59,216 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-27 03:54:59,216 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-27 03:54:59,216 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:59,217 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-27 03:54:59,217 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-27 03:54:59,219 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:59,219 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:54:59,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:59,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:59,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:59,219 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:54:59,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:59,219 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:54:59,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:59,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:54:59,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:59,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:59,221 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-27 03:54:59,221 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1052ce41[Count = 0] remaining members to acquire global barrier 2023-03-27 03:54:59,221 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-27 03:54:59,221 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:54:59,223 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:54:59,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:54:59,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:54:59,223 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:59,223 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:54:59,223 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-27 03:54:59,223 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-27 03:54:59,223 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:54:59,224 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-27 03:54:59,225 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:59,225 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-27 03:54:59,225 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:59,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:59,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:59,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:59,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:59,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:59,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:54:59,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:59,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:59,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:54:59,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:59,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:54:59,228 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-27 03:54:59,228 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-27 03:54:59,228 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:54:59,228 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:54:59,228 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-27 03:54:59,228 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:54:59,229 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:54:59,229 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:54:59,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:54:59,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:54:59,229 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:59,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:54:59,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:54:59,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:54:59,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:59,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:59,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:54:59,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:54:59,230 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:59,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:54:59,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:54:59,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:54:59,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:59,232 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:59,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:54:59,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:54:59,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:59,240 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:59,240 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:59,240 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:54:59,240 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:54:59,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:59,240 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:54:59,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:59,240 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-27 03:54:59,241 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:54:59,240 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:54:59,241 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:59,241 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:54:59,241 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:54:59,241 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:54:59,241 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-27 03:54:59,241 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:54:59,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:54:59,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:54:59,341 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:54:59,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:54:59,345 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:54:59,345 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:54:59,345 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:54:59,349 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:54:59,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-27 03:54:59,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-27 03:54:59,352 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-27 03:54:59,352 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:59,352 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-27 03:54:59,352 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-27 03:54:59,352 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:54:59,352 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:54:59,353 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:54:59,353 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:59,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:54:59,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:54:59,354 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:54:59,354 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:54:59,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:54:59,354 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:54:59,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:54:59,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:54:59,355 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-27 03:54:59,355 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-27 03:54:59,355 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-27 03:54:59,355 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-27 03:54:59,355 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:54:59,355 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-27 03:54:59,360 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:54:59,360 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. started... 2023-03-27 03:54:59,360 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:54:59,360 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. started... 2023-03-27 03:54:59,360 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing faf5e912b9b1c732d5f1903dc1242b64 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-27 03:54:59,360 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 08cac3032353cd1d586bb4d48f4a89b2 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-27 03:54:59,368 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-27 03:54:59,368 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:54:59,368 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:54:59,368 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. started... 2023-03-27 03:54:59,368 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing a96b2587b7874c72fc83ce1239d003a2 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-27 03:54:59,406 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032798ddb3a423784c4681291e1eae6554b1 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032798ddb3a423784c4681291e1eae6554b1 2023-03-27 03:54:59,799 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230327ce396922440a44a2abbfa143e96b7b4b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230327ce396922440a44a2abbfa143e96b7b4b 2023-03-27 03:54:59,805 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family1/5f23b1b075824f629deac284215eaab8 2023-03-27 03:54:59,808 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303274064056ec0ec4d9caf8180840bc868d6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303274064056ec0ec4d9caf8180840bc868d6 2023-03-27 03:54:59,816 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family1/750de3b9e4de457880e1f5d6a110e086 2023-03-27 03:55:00,214 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family1/1ee4d7888f284957b6d9006f42b915a9 2023-03-27 03:55:00,222 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230327254d12f544e646f1ae34d98ee00fc820 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230327254d12f544e646f1ae34d98ee00fc820 2023-03-27 03:55:00,232 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230327c37c951e5b074dd7b21fe060761f0697 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230327c37c951e5b074dd7b21fe060761f0697 2023-03-27 03:55:00,233 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032723d51c3ad3c94363a6fb22c0b5e70cf6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032723d51c3ad3c94363a6fb22c0b5e70cf6 2023-03-27 03:55:00,628 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family2/6d4e4ee3a2c44360afc16c2a2ba26705 2023-03-27 03:55:00,634 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family1/5f23b1b075824f629deac284215eaab8 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/5f23b1b075824f629deac284215eaab8 2023-03-27 03:55:00,638 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family2/cc6f9313ad64415aba03ab7d342199f9 2023-03-27 03:55:00,639 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family2/ce6a6eb80bb344489cd1e4201b2aa3ff 2023-03-27 03:55:00,639 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/5f23b1b075824f629deac284215eaab8, entries=4, sequenceid=13, filesize=5.4 K 2023-03-27 03:55:00,640 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family2/6d4e4ee3a2c44360afc16c2a2ba26705 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/6d4e4ee3a2c44360afc16c2a2ba26705 2023-03-27 03:55:00,647 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family1/1ee4d7888f284957b6d9006f42b915a9 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/1ee4d7888f284957b6d9006f42b915a9 2023-03-27 03:55:00,647 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family1/750de3b9e4de457880e1f5d6a110e086 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/750de3b9e4de457880e1f5d6a110e086 2023-03-27 03:55:00,651 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-27 03:55:00,651 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/6d4e4ee3a2c44360afc16c2a2ba26705, entries=2, sequenceid=13, filesize=5.1 K 2023-03-27 03:55:00,653 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 08cac3032353cd1d586bb4d48f4a89b2 in 1293ms, sequenceid=13, compaction requested=true 2023-03-27 03:55:00,653 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 08cac3032353cd1d586bb4d48f4a89b2: 2023-03-27 03:55:00,653 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:00,653 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-27 03:55:00,653 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/1ee4d7888f284957b6d9006f42b915a9, entries=4, sequenceid=13, filesize=5.4 K 2023-03-27 03:55:00,654 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family2/ce6a6eb80bb344489cd1e4201b2aa3ff as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/ce6a6eb80bb344489cd1e4201b2aa3ff 2023-03-27 03:55:00,656 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/750de3b9e4de457880e1f5d6a110e086, entries=4, sequenceid=13, filesize=5.4 K 2023-03-27 03:55:00,657 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family2/cc6f9313ad64415aba03ab7d342199f9 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/cc6f9313ad64415aba03ab7d342199f9 2023-03-27 03:55:00,674 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/ce6a6eb80bb344489cd1e4201b2aa3ff, entries=2, sequenceid=13, filesize=5.1 K 2023-03-27 03:55:00,675 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for faf5e912b9b1c732d5f1903dc1242b64 in 1315ms, sequenceid=13, compaction requested=true 2023-03-27 03:55:00,676 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for faf5e912b9b1c732d5f1903dc1242b64: 2023-03-27 03:55:00,676 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:00,676 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-27 03:55:00,685 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/cc6f9313ad64415aba03ab7d342199f9, entries=2, sequenceid=13, filesize=5.1 K 2023-03-27 03:55:00,686 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for a96b2587b7874c72fc83ce1239d003a2 in 1318ms, sequenceid=13, compaction requested=true 2023-03-27 03:55:00,686 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for a96b2587b7874c72fc83ce1239d003a2: 2023-03-27 03:55:00,686 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:00,686 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-27 03:55:00,687 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-27 03:55:00,687 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:00,687 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-27 03:55:00,687 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-27 03:55:00,690 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:00,690 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:00,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:00,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:00,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:00,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:00,691 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:00,692 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:00,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:00,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:55:00,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:00,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:00,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-27 03:55:00,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b8586f5[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:00,694 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-27 03:55:00,694 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:00,695 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:00,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:00,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:00,695 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-27 03:55:00,695 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-27 03:55:00,695 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:00,696 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:00,696 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:00,697 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:00,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:00,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:00,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:00,697 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-27 03:55:00,698 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:00,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:00,698 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-27 03:55:00,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:00,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:55:00,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:00,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:00,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:55:00,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:00,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:00,702 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-27 03:55:00,702 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-27 03:55:00,702 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:00,702 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:00,702 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-27 03:55:00,702 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:00,709 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:00,709 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:00,709 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:00,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:00,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:00,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:00,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:00,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:00,709 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:00,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:00,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:00,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:00,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:55:00,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:00,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:00,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:55:00,711 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:00,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:00,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:00,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:55:00,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:00,714 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:00,714 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:00,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:00,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:00,714 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:00,714 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:55:00,714 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:55:00,714 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:00,714 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:00,714 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:00,714 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:00,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:00,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:00,714 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-27 03:55:00,714 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:00,714 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-27 03:55:00,714 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:00,715 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:00,715 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:00,815 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:00,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:00,817 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:00,817 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:00,818 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:00,820 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:00,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-27 03:55:00,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-27 03:55:00,824 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-27 03:55:00,824 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:00,824 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-27 03:55:00,824 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-27 03:55:00,824 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:00,825 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:55:00,826 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:00,826 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:00,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:00,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:00,826 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:00,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:55:00,826 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:00,827 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:00,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:00,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:55:00,827 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-27 03:55:00,827 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-27 03:55:00,827 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-27 03:55:00,827 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-27 03:55:00,827 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:00,827 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-27 03:55:00,832 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:00,832 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. started... 2023-03-27 03:55:00,832 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:00,832 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. started... 2023-03-27 03:55:00,832 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing faf5e912b9b1c732d5f1903dc1242b64 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-27 03:55:00,832 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 08cac3032353cd1d586bb4d48f4a89b2 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-27 03:55:00,836 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-27 03:55:00,836 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:00,836 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:00,836 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. started... 2023-03-27 03:55:00,836 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing a96b2587b7874c72fc83ce1239d003a2 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-27 03:55:00,891 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032725ea9f84ed0e407b960174b9bfdd5af6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032725ea9f84ed0e407b960174b9bfdd5af6 2023-03-27 03:55:00,891 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032753f50e4eb60b43e48bdca7f0f84c5806 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023032753f50e4eb60b43e48bdca7f0f84c5806 2023-03-27 03:55:00,908 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family1/0de3d9f1ed40478c8ec24614c5d1e5c6 2023-03-27 03:55:00,929 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230327649d14f722054bc09df576f63c277f36 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230327649d14f722054bc09df576f63c277f36 2023-03-27 03:55:01,293 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303276ef5ec9cb35a4d9bb494381e01572d40 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303276ef5ec9cb35a4d9bb494381e01572d40 2023-03-27 03:55:01,300 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family1/dfa9ede98cc54cd9bb80e07c3fc3604c 2023-03-27 03:55:01,308 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family1/d867ce5762854f53a4b498bf51357bcd 2023-03-27 03:55:01,316 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230327f55424ed2c6443189f118f16f03c927c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230327f55424ed2c6443189f118f16f03c927c 2023-03-27 03:55:01,326 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family2/f11c1db0b6a44d3581c825a7593cd87f 2023-03-27 03:55:01,330 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family1/dfa9ede98cc54cd9bb80e07c3fc3604c as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/dfa9ede98cc54cd9bb80e07c3fc3604c 2023-03-27 03:55:01,336 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032791e57ad78612497db69ed50870fd14b6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023032791e57ad78612497db69ed50870fd14b6 2023-03-27 03:55:01,336 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/dfa9ede98cc54cd9bb80e07c3fc3604c, entries=4, sequenceid=17, filesize=5.4 K 2023-03-27 03:55:01,337 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family2/f11c1db0b6a44d3581c825a7593cd87f as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/f11c1db0b6a44d3581c825a7593cd87f 2023-03-27 03:55:01,342 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/f11c1db0b6a44d3581c825a7593cd87f, entries=2, sequenceid=17, filesize=5.1 K 2023-03-27 03:55:01,343 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for a96b2587b7874c72fc83ce1239d003a2 in 507ms, sequenceid=17, compaction requested=true 2023-03-27 03:55:01,343 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for a96b2587b7874c72fc83ce1239d003a2: 2023-03-27 03:55:01,343 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:01,343 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-27 03:55:01,343 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family2/dbf720c6cd314e418def7c5292224d93 2023-03-27 03:55:01,348 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family2/f5f824e90b9d45c7accbb5d4a6a9bd48 2023-03-27 03:55:01,349 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family1/0de3d9f1ed40478c8ec24614c5d1e5c6 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/0de3d9f1ed40478c8ec24614c5d1e5c6 2023-03-27 03:55:01,352 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family1/d867ce5762854f53a4b498bf51357bcd as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/d867ce5762854f53a4b498bf51357bcd 2023-03-27 03:55:01,353 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/0de3d9f1ed40478c8ec24614c5d1e5c6, entries=4, sequenceid=17, filesize=5.4 K 2023-03-27 03:55:01,354 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family2/dbf720c6cd314e418def7c5292224d93 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/dbf720c6cd314e418def7c5292224d93 2023-03-27 03:55:01,357 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/d867ce5762854f53a4b498bf51357bcd, entries=4, sequenceid=17, filesize=5.4 K 2023-03-27 03:55:01,357 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family2/f5f824e90b9d45c7accbb5d4a6a9bd48 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/f5f824e90b9d45c7accbb5d4a6a9bd48 2023-03-27 03:55:01,359 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/dbf720c6cd314e418def7c5292224d93, entries=2, sequenceid=17, filesize=5.1 K 2023-03-27 03:55:01,362 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for faf5e912b9b1c732d5f1903dc1242b64 in 530ms, sequenceid=17, compaction requested=true 2023-03-27 03:55:01,362 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for faf5e912b9b1c732d5f1903dc1242b64: 2023-03-27 03:55:01,362 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:01,362 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-27 03:55:01,364 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/f5f824e90b9d45c7accbb5d4a6a9bd48, entries=2, sequenceid=17, filesize=5.1 K 2023-03-27 03:55:01,365 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 08cac3032353cd1d586bb4d48f4a89b2 in 533ms, sequenceid=17, compaction requested=true 2023-03-27 03:55:01,365 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 08cac3032353cd1d586bb4d48f4a89b2: 2023-03-27 03:55:01,365 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:01,365 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-27 03:55:01,365 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-27 03:55:01,365 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,365 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-27 03:55:01,365 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-27 03:55:01,367 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:01,367 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:01,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:01,367 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:01,367 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:01,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:01,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:01,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:55:01,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:01,369 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-27 03:55:01,369 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41b25504[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:01,369 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-27 03:55:01,369 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:01,371 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:01,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:01,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:01,371 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-27 03:55:01,371 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-27 03:55:01,371 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,371 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:01,371 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:01,373 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:01,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:01,373 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-27 03:55:01,374 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:01,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:01,374 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-27 03:55:01,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:01,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:55:01,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:01,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:55:01,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:01,377 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-27 03:55:01,377 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:01,377 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:01,377 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-27 03:55:01,377 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-27 03:55:01,377 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:01,379 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:01,379 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:01,379 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:01,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:01,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:01,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:01,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:01,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:01,379 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:01,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:01,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:01,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:55:01,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:01,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:01,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:55:01,382 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:01,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:55:01,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,384 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:01,384 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:01,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:01,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:01,384 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,384 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:55:01,384 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:01,384 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:55:01,384 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,384 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:01,385 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:01,385 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:01,384 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-27 03:55:01,385 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:01,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:01,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:01,385 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:01,385 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:01,385 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-03-27 03:55:01,485 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:01,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:01,499 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1679889301486 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-27 03:55:01,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1679889301499 (current time:1679889301499). 2023-03-27 03:55:01,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-27 03:55:01,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] zookeeper.ReadOnlyZKClient(139): Connect 0x1188f756 to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:55:01,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@35c7f996, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:55:01,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-27 03:55:01,518 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34834, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-27 03:55:01,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1188f756 to 127.0.0.1:62246 2023-03-27 03:55:01,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:55:01,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-27 03:55:01,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1679889301486 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-27 03:55:01,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-27 03:55:01,531 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-27 03:55:01,532 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1679889901532 2023-03-27 03:55:01,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1679889301486 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-27 03:55:01,532 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1679889301486 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-27 03:55:01,533 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-27 03:55:01,534 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-27 03:55:01,534 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 4 msec 2023-03-27 03:55:01,536 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-27 03:55:01,537 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1679889901537 2023-03-27 03:55:01,538 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1679889301486 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:01,538 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-27 03:55:01,546 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1679889301486 2023-03-27 03:55:01,546 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1679889301486' 2023-03-27 03:55:01,546 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-27 03:55:01,546 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1679889301486' starting 'acquire' 2023-03-27 03:55:01,546 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1679889301486', kicking off acquire phase on members. 2023-03-27 03:55:01,547 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1679889301486 2023-03-27 03:55:01,547 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1679889301486 2023-03-27 03:55:01,548 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-27 03:55:01,548 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1679889301486/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-27 03:55:01,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-27 03:55:01,548 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1679889301486/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,548 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:01,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1679889301486 2023-03-27 03:55:01,549 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1679889301486 2023-03-27 03:55:01,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-03-27 03:55:01,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1679889301486 2023-03-27 03:55:01,549 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1679889301486 from table testCompactionWithHFileLink type FLUSH 2023-03-27 03:55:01,551 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1679889301486 2023-03-27 03:55:01,551 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1679889301486' with timeout 300000ms 2023-03-27 03:55:01,551 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-03-27 03:55:01,551 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1679889301486' starting 'acquire' stage 2023-03-27 03:55:01,552 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1679889301486' locally acquired 2023-03-27 03:55:01,552 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (snaptb-1679889301486) in zk 2023-03-27 03:55:01,554 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1679889301486/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,554 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1679889301486 2023-03-27 03:55:01,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1679889301486/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:01,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-27 03:55:01,555 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1679889301486 2023-03-27 03:55:01,555 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1679889301486' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:01,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:01,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:01,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679889301486 2023-03-27 03:55:01,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:01,556 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'snaptb-1679889301486' on coordinator 2023-03-27 03:55:01,556 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2817c312[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:01,556 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1679889301486' starting 'in-barrier' execution. 2023-03-27 03:55:01,556 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1679889301486 2023-03-27 03:55:01,557 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1679889301486 2023-03-27 03:55:01,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1679889301486 2023-03-27 03:55:01,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1679889301486 2023-03-27 03:55:01,557 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1679889301486' received 'reached' from coordinator. 2023-03-27 03:55:01,558 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1679889301486/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,558 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:01,558 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:01,559 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. started... 2023-03-27 03:55:01,559 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for faf5e912b9b1c732d5f1903dc1242b64: 2023-03-27 03:55:01,560 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:01,560 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. started... 2023-03-27 03:55:01,560 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for 08cac3032353cd1d586bb4d48f4a89b2: 2023-03-27 03:55:01,560 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64.' region-info for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,560 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2.' region-info for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,567 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-27 03:55:01,567 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-27 03:55:01,568 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-03-27 03:55:01,568 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:01,568 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-03-27 03:55:01,568 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. started... 2023-03-27 03:55:01,568 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for a96b2587b7874c72fc83ce1239d003a2: 2023-03-27 03:55:01,568 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2.' region-info for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,568 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-03-27 03:55:01,572 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/72ab7fd6b53b42a1ac7eed9e4568659d, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/a02d2e3e10344b54bba8a663ef52c568, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/750de3b9e4de457880e1f5d6a110e086, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/dfa9ede98cc54cd9bb80e07c3fc3604c] hfiles 2023-03-27 03:55:01,572 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/bdd2fd9286284264bb4230a42ecd2604, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/64b1111da52f413e9f2a26d0594a1069, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/5f23b1b075824f629deac284215eaab8, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/d867ce5762854f53a4b498bf51357bcd] hfiles 2023-03-27 03:55:01,572 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/bdd2fd9286284264bb4230a42ecd2604 for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,572 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/2a10d8329f234324aec49aae55e72329, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/5ae08f2ca4ae401f9e6acd9dfb0f908a, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/1ee4d7888f284957b6d9006f42b915a9, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/0de3d9f1ed40478c8ec24614c5d1e5c6] hfiles 2023-03-27 03:55:01,572 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/72ab7fd6b53b42a1ac7eed9e4568659d for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,572 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/2a10d8329f234324aec49aae55e72329 for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/64b1111da52f413e9f2a26d0594a1069 for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/5f23b1b075824f629deac284215eaab8 for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/a02d2e3e10344b54bba8a663ef52c568 for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/5ae08f2ca4ae401f9e6acd9dfb0f908a for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/750de3b9e4de457880e1f5d6a110e086 for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/1ee4d7888f284957b6d9006f42b915a9 for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/d867ce5762854f53a4b498bf51357bcd for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/0de3d9f1ed40478c8ec24614c5d1e5c6 for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/dfa9ede98cc54cd9bb80e07c3fc3604c for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/0af7c02db50945349a2e0c8532efb61d, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/7f0e82322b054d29ae0a7124a90065be, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/ce6a6eb80bb344489cd1e4201b2aa3ff, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/dbf720c6cd314e418def7c5292224d93] hfiles 2023-03-27 03:55:01,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/9ca06fff42104415a8c39a7223a455af, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/0ea65611c6ce4b80a7fa1a7b13e13710, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/cc6f9313ad64415aba03ab7d342199f9, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/f11c1db0b6a44d3581c825a7593cd87f] hfiles 2023-03-27 03:55:01,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/9ca06fff42104415a8c39a7223a455af for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/ba625f70d9564e14ac3e61abdbd37e22, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/4c5d382fe46d43cdb4bbba094ca5be83, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/6d4e4ee3a2c44360afc16c2a2ba26705, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/f5f824e90b9d45c7accbb5d4a6a9bd48] hfiles 2023-03-27 03:55:01,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/0ea65611c6ce4b80a7fa1a7b13e13710 for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/0af7c02db50945349a2e0c8532efb61d for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/cc6f9313ad64415aba03ab7d342199f9 for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/ba625f70d9564e14ac3e61abdbd37e22 for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/f11c1db0b6a44d3581c825a7593cd87f for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/4c5d382fe46d43cdb4bbba094ca5be83 for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/7f0e82322b054d29ae0a7124a90065be for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,577 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/ce6a6eb80bb344489cd1e4201b2aa3ff for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,577 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/dbf720c6cd314e418def7c5292224d93 for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,577 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/6d4e4ee3a2c44360afc16c2a2ba26705 for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,577 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/f5f824e90b9d45c7accbb5d4a6a9bd48 for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,595 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. completed. 2023-03-27 03:55:01,595 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:01,595 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. completed. 2023-03-27 03:55:01,596 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:01,596 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-03-27 03:55:01,596 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-03-27 03:55:01,598 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. completed. 2023-03-27 03:55:01,598 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:01,598 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-03-27 03:55:01,598 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-03-27 03:55:01,598 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,598 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1679889301486' locally completed 2023-03-27 03:55:01,598 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1679889301486' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:01,599 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1679889301486/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,599 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1679889301486' has notified controller of completion 2023-03-27 03:55:01,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1679889301486/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,600 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:01,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:01,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-27 03:55:01,600 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1679889301486' completed. 2023-03-27 03:55:01,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:01,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:01,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679889301486 2023-03-27 03:55:01,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:01,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679889301486 2023-03-27 03:55:01,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1679889301486' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:01,602 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'snaptb-1679889301486', counting down latch. Waiting for 0 more 2023-03-27 03:55:01,602 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1679889301486' execution completed 2023-03-27 03:55:01,602 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:01,602 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:01,602 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1679889301486 2023-03-27 03:55:01,602 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1679889301486including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-03-27 03:55:01,604 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1679889301486 2023-03-27 03:55:01,604 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1679889301486 2023-03-27 03:55:01,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1679889301486 2023-03-27 03:55:01,604 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-27 03:55:01,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1679889301486 2023-03-27 03:55:01,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1679889301486 2023-03-27 03:55:01,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:01,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-03-27 03:55:01,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-27 03:55:01,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-27 03:55:01,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:01,605 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1679889301486/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1679889301486 2023-03-27 03:55:01,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679889301486 2023-03-27 03:55:01,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:01,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679889301486 2023-03-27 03:55:01,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,606 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1679889301486/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:01,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1679889301486 2023-03-27 03:55:01,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,611 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-03-27 03:55:01,611 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-03-27 03:55:01,611 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:01,611 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1679889301486 2023-03-27 03:55:01,611 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-03-27 03:55:01,611 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1679889301486 2023-03-27 03:55:01,611 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-03-27 03:55:01,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-03-27 03:55:01,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-03-27 03:55:01,611 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1679889301486/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,611 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1679889301486 2023-03-27 03:55:01,611 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1679889301486 2023-03-27 03:55:01,612 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1679889301486/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:01,612 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1679889301486 2023-03-27 03:55:01,612 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1679889301486 2023-03-27 03:55:01,612 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1679889301486 2023-03-27 03:55:01,613 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230327059e3f0cf9874fe597ffaaa4cf3e8f2c, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032725ea9f84ed0e407b960174b9bfdd5af6, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303274064056ec0ec4d9caf8180840bc868d6, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230327a6b65f3b1d404ecc864089d09033c520, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303276ef5ec9cb35a4d9bb494381e01572d40, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032771c38408a9c64734b2b9a0fc42690dc7, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032798ddb3a423784c4681291e1eae6554b1, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230327c743126e94f14fd49aa60700919794b4, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303272e2c3641420440f5aac83f42c82b6a1d, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023032753f50e4eb60b43e48bdca7f0f84c5806, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023032762936666bd29429093aab0b1a3958593, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230327ce396922440a44a2abbfa143e96b7b4b] mob files 2023-03-27 03:55:01,613 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230327059e3f0cf9874fe597ffaaa4cf3e8f2c 2023-03-27 03:55:01,613 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032725ea9f84ed0e407b960174b9bfdd5af6 2023-03-27 03:55:01,613 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303274064056ec0ec4d9caf8180840bc868d6 2023-03-27 03:55:01,613 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230327a6b65f3b1d404ecc864089d09033c520 2023-03-27 03:55:01,613 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303276ef5ec9cb35a4d9bb494381e01572d40 2023-03-27 03:55:01,613 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032771c38408a9c64734b2b9a0fc42690dc7 2023-03-27 03:55:01,613 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032798ddb3a423784c4681291e1eae6554b1 2023-03-27 03:55:01,613 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230327c743126e94f14fd49aa60700919794b4 2023-03-27 03:55:01,613 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303272e2c3641420440f5aac83f42c82b6a1d 2023-03-27 03:55:01,613 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023032753f50e4eb60b43e48bdca7f0f84c5806 2023-03-27 03:55:01,613 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023032762936666bd29429093aab0b1a3958593 2023-03-27 03:55:01,614 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230327ce396922440a44a2abbfa143e96b7b4b 2023-03-27 03:55:01,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-03-27 03:55:01,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-03-27 03:55:01,615 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303272376dedadec0473e89d88d6fe66d697a, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230327649d14f722054bc09df576f63c277f36, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023032764acdbba253f4ebc945ddf3c8b1cdd81, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230327c37c951e5b074dd7b21fe060761f0697, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303271730ef134362450aac84bafaa4706420, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032723d51c3ad3c94363a6fb22c0b5e70cf6, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230327ece6da9680f44eda9d86b37189e09673, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230327f55424ed2c6443189f118f16f03c927c, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230327254d12f544e646f1ae34d98ee00fc820, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303275668c83e5a6b43feb6e5d72e713bfa77, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023032791e57ad78612497db69ed50870fd14b6, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230327fa6e6618f562408ca885c0890360c478] mob files 2023-03-27 03:55:01,615 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303272376dedadec0473e89d88d6fe66d697a 2023-03-27 03:55:01,615 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230327649d14f722054bc09df576f63c277f36 2023-03-27 03:55:01,615 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023032764acdbba253f4ebc945ddf3c8b1cdd81 2023-03-27 03:55:01,615 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230327c37c951e5b074dd7b21fe060761f0697 2023-03-27 03:55:01,615 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303271730ef134362450aac84bafaa4706420 2023-03-27 03:55:01,615 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032723d51c3ad3c94363a6fb22c0b5e70cf6 2023-03-27 03:55:01,615 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230327ece6da9680f44eda9d86b37189e09673 2023-03-27 03:55:01,615 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230327f55424ed2c6443189f118f16f03c927c 2023-03-27 03:55:01,615 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230327254d12f544e646f1ae34d98ee00fc820 2023-03-27 03:55:01,615 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303275668c83e5a6b43feb6e5d72e713bfa77 2023-03-27 03:55:01,615 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023032791e57ad78612497db69ed50870fd14b6 2023-03-27 03:55:01,615 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230327fa6e6618f562408ca885c0890360c478 2023-03-27 03:55:01,628 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1679889301486 2023-03-27 03:55:01,629 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.hbase-snapshot/.tmp/snaptb-1679889301486 2023-03-27 03:55:01,638 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-27 03:55:01,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1679889301486 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-27 03:55:01,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1679889301486 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-27 03:55:01,643 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-27 03:55:01,714 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.hbase-snapshot/.tmp/snaptb-1679889301486 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.hbase-snapshot/snaptb-1679889301486 2023-03-27 03:55:01,843 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-27 03:55:01,844 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1679889301486 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-27 03:55:01,844 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1679889301486 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-27 03:55:01,844 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-27 03:55:02,145 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-27 03:55:02,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1679889301486 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-27 03:55:02,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1679889301486 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-27 03:55:02,146 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-27 03:55:02,187 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1679889301486 of table testCompactionWithHFileLink completed 2023-03-27 03:55:02,187 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.hbase-snapshot/.tmp/snaptb-1679889301486 2023-03-27 03:55:02,187 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.hbase-snapshot/.tmp/snaptb-1679889301486 2023-03-27 03:55:02,188 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1679889301486 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1679889301532Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1679889301627 (+95 ms)Consolidate snapshot: snaptb-1679889301486 at 1679889301628 (+1 ms)Loading Region manifests for snaptb-1679889301486 at 1679889301628Writing data manifest for snaptb-1679889301486 at 1679889301646 (+18 ms)Verifying snapshot: snaptb-1679889301486 at 1679889301673 (+27 ms)Snapshot snaptb-1679889301486 of table testCompactionWithHFileLink completed at 1679889302187 (+514 ms) 2023-03-27 03:55:02,189 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-03-27 03:55:02,190 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 655 msec 2023-03-27 03:55:02,646 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-27 03:55:02,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1679889301486 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-27 03:55:02,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1679889301486 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-27 03:55:02,688 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:02,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-03-27 03:55:02,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-03-27 03:55:02,690 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-03-27 03:55:02,690 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:02,690 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-03-27 03:55:02,690 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-03-27 03:55:02,690 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:02,690 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:55:02,692 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:02,692 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:02,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:02,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:02,693 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:55:02,693 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:02,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:02,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:02,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:55:02,694 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-03-27 03:55:02,694 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-03-27 03:55:02,694 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-03-27 03:55:02,694 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-03-27 03:55:02,694 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:02,694 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-03-27 03:55:02,696 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:02,696 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. started... 2023-03-27 03:55:02,696 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing faf5e912b9b1c732d5f1903dc1242b64 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-27 03:55:02,704 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:02,704 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. started... 2023-03-27 03:55:02,704 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 08cac3032353cd1d586bb4d48f4a89b2 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-27 03:55:02,705 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-27 03:55:02,705 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:02,705 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:02,705 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. started... 2023-03-27 03:55:02,705 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing a96b2587b7874c72fc83ce1239d003a2 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-27 03:55:02,748 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family1/5580f744bb354c5b85be175401a61058 2023-03-27 03:55:02,753 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5580f744bb354c5b85be175401a61058 2023-03-27 03:55:02,753 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family1/ac5fd9b241434561839a4c8cd2fc3829 2023-03-27 03:55:02,757 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ac5fd9b241434561839a4c8cd2fc3829 2023-03-27 03:55:03,149 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family1/f8cb5ea4b36c40a2b14ece3edc8c7d0b 2023-03-27 03:55:03,153 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f8cb5ea4b36c40a2b14ece3edc8c7d0b 2023-03-27 03:55:03,568 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family2/b12ae52ee8f4470cab4246ad30e4d45d 2023-03-27 03:55:03,572 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b12ae52ee8f4470cab4246ad30e4d45d 2023-03-27 03:55:03,573 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family1/f8cb5ea4b36c40a2b14ece3edc8c7d0b as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/f8cb5ea4b36c40a2b14ece3edc8c7d0b 2023-03-27 03:55:03,588 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family2/e2d05847ceb545feabce03099c5d9c30 2023-03-27 03:55:03,590 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family2/f501f6a990cb44498e1662ccb3456880 2023-03-27 03:55:03,592 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f8cb5ea4b36c40a2b14ece3edc8c7d0b 2023-03-27 03:55:03,592 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/f8cb5ea4b36c40a2b14ece3edc8c7d0b, entries=3, sequenceid=24, filesize=5.0 K 2023-03-27 03:55:03,595 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e2d05847ceb545feabce03099c5d9c30 2023-03-27 03:55:03,595 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family2/b12ae52ee8f4470cab4246ad30e4d45d as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/b12ae52ee8f4470cab4246ad30e4d45d 2023-03-27 03:55:03,596 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family1/ac5fd9b241434561839a4c8cd2fc3829 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/ac5fd9b241434561839a4c8cd2fc3829 2023-03-27 03:55:03,600 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b12ae52ee8f4470cab4246ad30e4d45d 2023-03-27 03:55:03,600 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/b12ae52ee8f4470cab4246ad30e4d45d, entries=1, sequenceid=24, filesize=5.0 K 2023-03-27 03:55:03,601 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 08cac3032353cd1d586bb4d48f4a89b2 in 897ms, sequenceid=24, compaction requested=true 2023-03-27 03:55:03,601 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 08cac3032353cd1d586bb4d48f4a89b2: 2023-03-27 03:55:03,602 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:03,602 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-27 03:55:03,602 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ac5fd9b241434561839a4c8cd2fc3829 2023-03-27 03:55:03,602 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/ac5fd9b241434561839a4c8cd2fc3829, entries=3, sequenceid=24, filesize=5.0 K 2023-03-27 03:55:03,603 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family2/e2d05847ceb545feabce03099c5d9c30 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/e2d05847ceb545feabce03099c5d9c30 2023-03-27 03:55:03,608 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e2d05847ceb545feabce03099c5d9c30 2023-03-27 03:55:03,608 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/e2d05847ceb545feabce03099c5d9c30, entries=1, sequenceid=24, filesize=5.0 K 2023-03-27 03:55:03,608 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f501f6a990cb44498e1662ccb3456880 2023-03-27 03:55:03,609 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for a96b2587b7874c72fc83ce1239d003a2 in 904ms, sequenceid=24, compaction requested=true 2023-03-27 03:55:03,609 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family1/5580f744bb354c5b85be175401a61058 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/5580f744bb354c5b85be175401a61058 2023-03-27 03:55:03,609 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for a96b2587b7874c72fc83ce1239d003a2: 2023-03-27 03:55:03,609 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:03,609 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-27 03:55:03,613 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5580f744bb354c5b85be175401a61058 2023-03-27 03:55:03,613 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/5580f744bb354c5b85be175401a61058, entries=3, sequenceid=24, filesize=5.0 K 2023-03-27 03:55:03,614 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family2/f501f6a990cb44498e1662ccb3456880 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/f501f6a990cb44498e1662ccb3456880 2023-03-27 03:55:03,619 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f501f6a990cb44498e1662ccb3456880 2023-03-27 03:55:03,619 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/f501f6a990cb44498e1662ccb3456880, entries=1, sequenceid=24, filesize=5.0 K 2023-03-27 03:55:03,620 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for faf5e912b9b1c732d5f1903dc1242b64 in 924ms, sequenceid=24, compaction requested=true 2023-03-27 03:55:03,620 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for faf5e912b9b1c732d5f1903dc1242b64: 2023-03-27 03:55:03,620 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:03,620 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-27 03:55:03,620 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-27 03:55:03,620 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:03,620 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-03-27 03:55:03,620 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-03-27 03:55:03,622 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:03,622 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:03,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:03,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:03,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:03,623 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:03,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:03,623 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:03,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:03,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:55:03,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:03,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:03,625 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-03-27 03:55:03,625 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2eb13097[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:03,625 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-03-27 03:55:03,625 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:03,626 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:03,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:03,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:03,626 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-03-27 03:55:03,626 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-03-27 03:55:03,626 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:03,627 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:03,627 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:03,632 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:03,632 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-03-27 03:55:03,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:03,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:03,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:03,632 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:03,632 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-03-27 03:55:03,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:03,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:03,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:55:03,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:03,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:03,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:55:03,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:03,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:03,635 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-03-27 03:55:03,635 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-03-27 03:55:03,635 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:03,635 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:03,635 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-03-27 03:55:03,635 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:03,637 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:03,637 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:03,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:03,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:03,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:03,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:03,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:03,637 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:03,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:03,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:03,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:03,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:03,640 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:03,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:55:03,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:03,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:55:03,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:03,642 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:03,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:03,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-03-27 03:55:03,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:03,647 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:03,647 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:03,647 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:55:03,647 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:03,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:03,647 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-03-27 03:55:03,647 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:03,647 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:03,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:03,647 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-03-27 03:55:03,647 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:03,647 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:03,647 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-03-27 03:55:03,647 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-03-27 03:55:03,647 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:03,648 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:03,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:03,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:03,748 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:03,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:03,750 DEBUG [Listener at localhost/36131] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-27 03:55:03,750 DEBUG [Listener at localhost/36131] regionserver.HStore(1912): faf5e912b9b1c732d5f1903dc1242b64/family1 is initiating major compaction (all files) 2023-03-27 03:55:03,750 INFO [Listener at localhost/36131] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-27 03:55:03,750 INFO [Listener at localhost/36131] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-27 03:55:03,751 INFO [Listener at localhost/36131] regionserver.HRegion(2259): Starting compaction of faf5e912b9b1c732d5f1903dc1242b64/family1 in testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:03,751 INFO [Listener at localhost/36131] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/2a10d8329f234324aec49aae55e72329, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/5ae08f2ca4ae401f9e6acd9dfb0f908a, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/1ee4d7888f284957b6d9006f42b915a9, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/0de3d9f1ed40478c8ec24614c5d1e5c6, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/5580f744bb354c5b85be175401a61058] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp, totalSize=26.7 K 2023-03-27 03:55:03,752 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 2a10d8329f234324aec49aae55e72329, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889296065 2023-03-27 03:55:03,752 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 5ae08f2ca4ae401f9e6acd9dfb0f908a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889297513 2023-03-27 03:55:03,753 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 1ee4d7888f284957b6d9006f42b915a9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889299345 2023-03-27 03:55:03,753 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 0de3d9f1ed40478c8ec24614c5d1e5c6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889300817 2023-03-27 03:55:03,754 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 5580f744bb354c5b85be175401a61058, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-27 03:55:03,769 INFO [Listener at localhost/36131] throttle.PressureAwareThroughputController(145): faf5e912b9b1c732d5f1903dc1242b64#family1#compaction#108 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:03,786 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023032776089d93172f41d680bf3b47f5f12fdb_del 2023-03-27 03:55:03,786 INFO [Listener at localhost/36131] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032776089d93172f41d680bf3b47f5f12fdb_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032776089d93172f41d680bf3b47f5f12fdb_del 2023-03-27 03:55:04,204 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 86a5c0a8e88b46c0836ae41a4acee1a1 2023-03-27 03:55:04,205 DEBUG [Listener at localhost/36131] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family1/86a5c0a8e88b46c0836ae41a4acee1a1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/86a5c0a8e88b46c0836ae41a4acee1a1 2023-03-27 03:55:04,210 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 86a5c0a8e88b46c0836ae41a4acee1a1 2023-03-27 03:55:04,211 INFO [Listener at localhost/36131] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in faf5e912b9b1c732d5f1903dc1242b64/family1 of faf5e912b9b1c732d5f1903dc1242b64 into 86a5c0a8e88b46c0836ae41a4acee1a1(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:55:04,211 DEBUG [Listener at localhost/36131] regionserver.HRegion(2289): Compaction status journal for faf5e912b9b1c732d5f1903dc1242b64: 2023-03-27 03:55:04,211 DEBUG [Listener at localhost/36131] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-27 03:55:04,211 DEBUG [Listener at localhost/36131] regionserver.HStore(1912): faf5e912b9b1c732d5f1903dc1242b64/family2 is initiating major compaction (all files) 2023-03-27 03:55:04,211 INFO [Listener at localhost/36131] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-27 03:55:04,212 INFO [Listener at localhost/36131] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-27 03:55:04,212 INFO [Listener at localhost/36131] regionserver.HRegion(2259): Starting compaction of faf5e912b9b1c732d5f1903dc1242b64/family2 in testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:04,213 INFO [Listener at localhost/36131] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/0af7c02db50945349a2e0c8532efb61d, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/7f0e82322b054d29ae0a7124a90065be, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/ce6a6eb80bb344489cd1e4201b2aa3ff, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/dbf720c6cd314e418def7c5292224d93, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/f501f6a990cb44498e1662ccb3456880] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp, totalSize=25.4 K 2023-03-27 03:55:04,213 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 0af7c02db50945349a2e0c8532efb61d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889296065 2023-03-27 03:55:04,214 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 7f0e82322b054d29ae0a7124a90065be, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889297513 2023-03-27 03:55:04,214 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting ce6a6eb80bb344489cd1e4201b2aa3ff, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889299345 2023-03-27 03:55:04,215 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting dbf720c6cd314e418def7c5292224d93, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889300817 2023-03-27 03:55:04,215 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting f501f6a990cb44498e1662ccb3456880, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-27 03:55:04,240 INFO [Listener at localhost/36131] throttle.PressureAwareThroughputController(145): faf5e912b9b1c732d5f1903dc1242b64#family2#compaction#109 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:04,253 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230327fa03c6934b074c4a9b33a6b8521eb183_del 2023-03-27 03:55:04,253 INFO [Listener at localhost/36131] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230327fa03c6934b074c4a9b33a6b8521eb183_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230327fa03c6934b074c4a9b33a6b8521eb183_del 2023-03-27 03:55:04,294 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/2a10d8329f234324aec49aae55e72329, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/5ae08f2ca4ae401f9e6acd9dfb0f908a, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/1ee4d7888f284957b6d9006f42b915a9, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/0de3d9f1ed40478c8ec24614c5d1e5c6, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/5580f744bb354c5b85be175401a61058] to archive 2023-03-27 03:55:04,295 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:04,296 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/2a10d8329f234324aec49aae55e72329 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/2a10d8329f234324aec49aae55e72329 2023-03-27 03:55:04,298 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/5ae08f2ca4ae401f9e6acd9dfb0f908a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/5ae08f2ca4ae401f9e6acd9dfb0f908a 2023-03-27 03:55:04,299 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/1ee4d7888f284957b6d9006f42b915a9 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/1ee4d7888f284957b6d9006f42b915a9 2023-03-27 03:55:04,300 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/0de3d9f1ed40478c8ec24614c5d1e5c6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/0de3d9f1ed40478c8ec24614c5d1e5c6 2023-03-27 03:55:04,301 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/5580f744bb354c5b85be175401a61058 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/5580f744bb354c5b85be175401a61058 2023-03-27 03:55:04,668 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4cda7786940b4cf58a5177114b209b81 2023-03-27 03:55:04,669 DEBUG [Listener at localhost/36131] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family2/4cda7786940b4cf58a5177114b209b81 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/4cda7786940b4cf58a5177114b209b81 2023-03-27 03:55:04,672 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4cda7786940b4cf58a5177114b209b81 2023-03-27 03:55:04,673 INFO [Listener at localhost/36131] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in faf5e912b9b1c732d5f1903dc1242b64/family2 of faf5e912b9b1c732d5f1903dc1242b64 into 4cda7786940b4cf58a5177114b209b81(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:55:04,673 DEBUG [Listener at localhost/36131] regionserver.HRegion(2289): Compaction status journal for faf5e912b9b1c732d5f1903dc1242b64: 2023-03-27 03:55:04,673 DEBUG [Listener at localhost/36131] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-27 03:55:04,674 DEBUG [Listener at localhost/36131] regionserver.HStore(1912): 08cac3032353cd1d586bb4d48f4a89b2/family1 is initiating major compaction (all files) 2023-03-27 03:55:04,674 INFO [Listener at localhost/36131] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-27 03:55:04,674 INFO [Listener at localhost/36131] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-27 03:55:04,675 INFO [Listener at localhost/36131] regionserver.HRegion(2259): Starting compaction of 08cac3032353cd1d586bb4d48f4a89b2/family1 in testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:04,675 INFO [Listener at localhost/36131] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/bdd2fd9286284264bb4230a42ecd2604, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/64b1111da52f413e9f2a26d0594a1069, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/5f23b1b075824f629deac284215eaab8, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/d867ce5762854f53a4b498bf51357bcd, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/f8cb5ea4b36c40a2b14ece3edc8c7d0b] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp, totalSize=26.7 K 2023-03-27 03:55:04,675 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting bdd2fd9286284264bb4230a42ecd2604, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889296063 2023-03-27 03:55:04,676 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 64b1111da52f413e9f2a26d0594a1069, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889297513 2023-03-27 03:55:04,676 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 5f23b1b075824f629deac284215eaab8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889299345 2023-03-27 03:55:04,677 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting d867ce5762854f53a4b498bf51357bcd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889300817 2023-03-27 03:55:04,677 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting f8cb5ea4b36c40a2b14ece3edc8c7d0b, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-27 03:55:04,689 INFO [Listener at localhost/36131] throttle.PressureAwareThroughputController(145): 08cac3032353cd1d586bb4d48f4a89b2#family1#compaction#110 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:04,693 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/0af7c02db50945349a2e0c8532efb61d, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/7f0e82322b054d29ae0a7124a90065be, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/ce6a6eb80bb344489cd1e4201b2aa3ff, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/dbf720c6cd314e418def7c5292224d93, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/f501f6a990cb44498e1662ccb3456880] to archive 2023-03-27 03:55:04,694 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:04,695 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/0af7c02db50945349a2e0c8532efb61d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/0af7c02db50945349a2e0c8532efb61d 2023-03-27 03:55:04,697 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/7f0e82322b054d29ae0a7124a90065be to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/7f0e82322b054d29ae0a7124a90065be 2023-03-27 03:55:04,698 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/ce6a6eb80bb344489cd1e4201b2aa3ff to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/ce6a6eb80bb344489cd1e4201b2aa3ff 2023-03-27 03:55:04,700 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/dbf720c6cd314e418def7c5292224d93 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/dbf720c6cd314e418def7c5292224d93 2023-03-27 03:55:04,701 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/f501f6a990cb44498e1662ccb3456880 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/f501f6a990cb44498e1662ccb3456880 2023-03-27 03:55:05,109 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303271e65a5e07ebb4872901644d2a7d60eb4_del 2023-03-27 03:55:05,109 INFO [Listener at localhost/36131] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303271e65a5e07ebb4872901644d2a7d60eb4_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303271e65a5e07ebb4872901644d2a7d60eb4_del 2023-03-27 03:55:05,925 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bdaefea9d97441fa87a50e8f064b4e2b 2023-03-27 03:55:05,926 DEBUG [Listener at localhost/36131] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family1/bdaefea9d97441fa87a50e8f064b4e2b as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/bdaefea9d97441fa87a50e8f064b4e2b 2023-03-27 03:55:05,930 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bdaefea9d97441fa87a50e8f064b4e2b 2023-03-27 03:55:05,931 INFO [Listener at localhost/36131] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 08cac3032353cd1d586bb4d48f4a89b2/family1 of 08cac3032353cd1d586bb4d48f4a89b2 into bdaefea9d97441fa87a50e8f064b4e2b(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-27 03:55:05,931 DEBUG [Listener at localhost/36131] regionserver.HRegion(2289): Compaction status journal for 08cac3032353cd1d586bb4d48f4a89b2: 2023-03-27 03:55:05,931 DEBUG [Listener at localhost/36131] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-27 03:55:05,931 DEBUG [Listener at localhost/36131] regionserver.HStore(1912): 08cac3032353cd1d586bb4d48f4a89b2/family2 is initiating major compaction (all files) 2023-03-27 03:55:05,931 INFO [Listener at localhost/36131] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-27 03:55:05,931 INFO [Listener at localhost/36131] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-27 03:55:05,932 INFO [Listener at localhost/36131] regionserver.HRegion(2259): Starting compaction of 08cac3032353cd1d586bb4d48f4a89b2/family2 in testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:05,932 INFO [Listener at localhost/36131] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/ba625f70d9564e14ac3e61abdbd37e22, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/4c5d382fe46d43cdb4bbba094ca5be83, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/6d4e4ee3a2c44360afc16c2a2ba26705, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/f5f824e90b9d45c7accbb5d4a6a9bd48, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/b12ae52ee8f4470cab4246ad30e4d45d] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp, totalSize=25.4 K 2023-03-27 03:55:05,933 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting ba625f70d9564e14ac3e61abdbd37e22, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889296063 2023-03-27 03:55:05,933 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 4c5d382fe46d43cdb4bbba094ca5be83, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889297513 2023-03-27 03:55:05,934 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 6d4e4ee3a2c44360afc16c2a2ba26705, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889299345 2023-03-27 03:55:05,935 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting f5f824e90b9d45c7accbb5d4a6a9bd48, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889300817 2023-03-27 03:55:05,936 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting b12ae52ee8f4470cab4246ad30e4d45d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-27 03:55:05,947 INFO [Listener at localhost/36131] throttle.PressureAwareThroughputController(145): 08cac3032353cd1d586bb4d48f4a89b2#family2#compaction#111 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:05,993 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/bdd2fd9286284264bb4230a42ecd2604, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/64b1111da52f413e9f2a26d0594a1069, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/5f23b1b075824f629deac284215eaab8, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/d867ce5762854f53a4b498bf51357bcd, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/f8cb5ea4b36c40a2b14ece3edc8c7d0b] to archive 2023-03-27 03:55:05,994 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:05,995 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/bdd2fd9286284264bb4230a42ecd2604 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/bdd2fd9286284264bb4230a42ecd2604 2023-03-27 03:55:05,996 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/64b1111da52f413e9f2a26d0594a1069 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/64b1111da52f413e9f2a26d0594a1069 2023-03-27 03:55:05,997 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/5f23b1b075824f629deac284215eaab8 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/5f23b1b075824f629deac284215eaab8 2023-03-27 03:55:05,998 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/d867ce5762854f53a4b498bf51357bcd to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/d867ce5762854f53a4b498bf51357bcd 2023-03-27 03:55:05,999 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/f8cb5ea4b36c40a2b14ece3edc8c7d0b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/f8cb5ea4b36c40a2b14ece3edc8c7d0b 2023-03-27 03:55:06,355 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230327bba3cebb376f4d6d8e701bac9fe6e212_del 2023-03-27 03:55:06,355 INFO [Listener at localhost/36131] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230327bba3cebb376f4d6d8e701bac9fe6e212_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230327bba3cebb376f4d6d8e701bac9fe6e212_del 2023-03-27 03:55:07,168 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9111949b017149efbcb1733fd85bbcf3 2023-03-27 03:55:07,169 DEBUG [Listener at localhost/36131] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family2/9111949b017149efbcb1733fd85bbcf3 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/9111949b017149efbcb1733fd85bbcf3 2023-03-27 03:55:07,173 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9111949b017149efbcb1733fd85bbcf3 2023-03-27 03:55:07,174 INFO [Listener at localhost/36131] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 08cac3032353cd1d586bb4d48f4a89b2/family2 of 08cac3032353cd1d586bb4d48f4a89b2 into 9111949b017149efbcb1733fd85bbcf3(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-27 03:55:07,174 DEBUG [Listener at localhost/36131] regionserver.HRegion(2289): Compaction status journal for 08cac3032353cd1d586bb4d48f4a89b2: 2023-03-27 03:55:07,175 DEBUG [Listener at localhost/36131] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-27 03:55:07,175 DEBUG [Listener at localhost/36131] regionserver.HStore(1912): a96b2587b7874c72fc83ce1239d003a2/family1 is initiating major compaction (all files) 2023-03-27 03:55:07,175 INFO [Listener at localhost/36131] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-27 03:55:07,175 INFO [Listener at localhost/36131] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-27 03:55:07,176 INFO [Listener at localhost/36131] regionserver.HRegion(2259): Starting compaction of a96b2587b7874c72fc83ce1239d003a2/family1 in testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:07,176 INFO [Listener at localhost/36131] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/72ab7fd6b53b42a1ac7eed9e4568659d, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/a02d2e3e10344b54bba8a663ef52c568, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/750de3b9e4de457880e1f5d6a110e086, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/dfa9ede98cc54cd9bb80e07c3fc3604c, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/ac5fd9b241434561839a4c8cd2fc3829] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp, totalSize=26.7 K 2023-03-27 03:55:07,176 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 72ab7fd6b53b42a1ac7eed9e4568659d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889296066 2023-03-27 03:55:07,177 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting a02d2e3e10344b54bba8a663ef52c568, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889297513 2023-03-27 03:55:07,177 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 750de3b9e4de457880e1f5d6a110e086, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889299345 2023-03-27 03:55:07,178 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting dfa9ede98cc54cd9bb80e07c3fc3604c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889300818 2023-03-27 03:55:07,178 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting ac5fd9b241434561839a4c8cd2fc3829, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-27 03:55:07,189 INFO [Listener at localhost/36131] throttle.PressureAwareThroughputController(145): a96b2587b7874c72fc83ce1239d003a2#family1#compaction#112 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:07,193 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/ba625f70d9564e14ac3e61abdbd37e22, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/4c5d382fe46d43cdb4bbba094ca5be83, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/6d4e4ee3a2c44360afc16c2a2ba26705, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/f5f824e90b9d45c7accbb5d4a6a9bd48, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/b12ae52ee8f4470cab4246ad30e4d45d] to archive 2023-03-27 03:55:07,194 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:07,195 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/ba625f70d9564e14ac3e61abdbd37e22 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/ba625f70d9564e14ac3e61abdbd37e22 2023-03-27 03:55:07,196 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/4c5d382fe46d43cdb4bbba094ca5be83 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/4c5d382fe46d43cdb4bbba094ca5be83 2023-03-27 03:55:07,197 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/6d4e4ee3a2c44360afc16c2a2ba26705 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/6d4e4ee3a2c44360afc16c2a2ba26705 2023-03-27 03:55:07,198 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/f5f824e90b9d45c7accbb5d4a6a9bd48 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/f5f824e90b9d45c7accbb5d4a6a9bd48 2023-03-27 03:55:07,199 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/b12ae52ee8f4470cab4246ad30e4d45d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/b12ae52ee8f4470cab4246ad30e4d45d 2023-03-27 03:55:07,597 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230327a5f6fb5e1d4c4ca2a673355b05d21eb0_del 2023-03-27 03:55:07,597 INFO [Listener at localhost/36131] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230327a5f6fb5e1d4c4ca2a673355b05d21eb0_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230327a5f6fb5e1d4c4ca2a673355b05d21eb0_del 2023-03-27 03:55:08,411 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 04750adcab514db382f051848edf4589 2023-03-27 03:55:08,411 DEBUG [Listener at localhost/36131] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family1/04750adcab514db382f051848edf4589 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/04750adcab514db382f051848edf4589 2023-03-27 03:55:08,436 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-27 03:55:08,437 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x101a97d7 to 127.0.0.1:62246 2023-03-27 03:55:08,437 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:55:08,437 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-27 03:55:08,437 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6cfd95fd to 127.0.0.1:62246 2023-03-27 03:55:08,437 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:55:08,437 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-27 03:55:08,437 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x31d030e0 to 127.0.0.1:62246 2023-03-27 03:55:08,437 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:55:08,437 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x13634b6c to 127.0.0.1:62246 2023-03-27 03:55:08,437 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:55:08,438 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 04750adcab514db382f051848edf4589 2023-03-27 03:55:08,439 INFO [Listener at localhost/36131] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a96b2587b7874c72fc83ce1239d003a2/family1 of a96b2587b7874c72fc83ce1239d003a2 into 04750adcab514db382f051848edf4589(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-27 03:55:08,439 DEBUG [Listener at localhost/36131] regionserver.HRegion(2289): Compaction status journal for a96b2587b7874c72fc83ce1239d003a2: 2023-03-27 03:55:08,439 DEBUG [Listener at localhost/36131] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-27 03:55:08,439 DEBUG [Listener at localhost/36131] regionserver.HStore(1912): a96b2587b7874c72fc83ce1239d003a2/family2 is initiating major compaction (all files) 2023-03-27 03:55:08,440 INFO [Listener at localhost/36131] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-27 03:55:08,440 INFO [Listener at localhost/36131] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-27 03:55:08,440 INFO [Listener at localhost/36131] regionserver.HRegion(2259): Starting compaction of a96b2587b7874c72fc83ce1239d003a2/family2 in testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:08,441 INFO [Listener at localhost/36131] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/9ca06fff42104415a8c39a7223a455af, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/0ea65611c6ce4b80a7fa1a7b13e13710, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/cc6f9313ad64415aba03ab7d342199f9, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/f11c1db0b6a44d3581c825a7593cd87f, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/e2d05847ceb545feabce03099c5d9c30] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp, totalSize=25.4 K 2023-03-27 03:55:08,441 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 9ca06fff42104415a8c39a7223a455af, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889296066 2023-03-27 03:55:08,442 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 0ea65611c6ce4b80a7fa1a7b13e13710, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889297513 2023-03-27 03:55:08,442 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting cc6f9313ad64415aba03ab7d342199f9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889299345 2023-03-27 03:55:08,443 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting f11c1db0b6a44d3581c825a7593cd87f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889300818 2023-03-27 03:55:08,443 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting e2d05847ceb545feabce03099c5d9c30, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-03-27 03:55:08,456 INFO [Listener at localhost/36131] throttle.PressureAwareThroughputController(145): a96b2587b7874c72fc83ce1239d003a2#family2#compaction#113 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:08,493 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/72ab7fd6b53b42a1ac7eed9e4568659d, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/a02d2e3e10344b54bba8a663ef52c568, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/750de3b9e4de457880e1f5d6a110e086, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/dfa9ede98cc54cd9bb80e07c3fc3604c, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/ac5fd9b241434561839a4c8cd2fc3829] to archive 2023-03-27 03:55:08,494 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:08,495 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/72ab7fd6b53b42a1ac7eed9e4568659d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/72ab7fd6b53b42a1ac7eed9e4568659d 2023-03-27 03:55:08,496 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/a02d2e3e10344b54bba8a663ef52c568 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/a02d2e3e10344b54bba8a663ef52c568 2023-03-27 03:55:08,497 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/750de3b9e4de457880e1f5d6a110e086 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/750de3b9e4de457880e1f5d6a110e086 2023-03-27 03:55:08,498 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/dfa9ede98cc54cd9bb80e07c3fc3604c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/dfa9ede98cc54cd9bb80e07c3fc3604c 2023-03-27 03:55:08,499 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/ac5fd9b241434561839a4c8cd2fc3829 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/ac5fd9b241434561839a4c8cd2fc3829 2023-03-27 03:55:08,864 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303272600c42cea424fbf8cf9829ac2850b74_del 2023-03-27 03:55:08,864 INFO [Listener at localhost/36131] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303272600c42cea424fbf8cf9829ac2850b74_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303272600c42cea424fbf8cf9829ac2850b74_del 2023-03-27 03:55:09,677 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc2f5b9948654dc2995b26ae720209af 2023-03-27 03:55:09,678 DEBUG [Listener at localhost/36131] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family2/dc2f5b9948654dc2995b26ae720209af as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/dc2f5b9948654dc2995b26ae720209af 2023-03-27 03:55:09,682 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc2f5b9948654dc2995b26ae720209af 2023-03-27 03:55:09,683 INFO [Listener at localhost/36131] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a96b2587b7874c72fc83ce1239d003a2/family2 of a96b2587b7874c72fc83ce1239d003a2 into dc2f5b9948654dc2995b26ae720209af(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-27 03:55:09,683 DEBUG [Listener at localhost/36131] regionserver.HRegion(2289): Compaction status journal for a96b2587b7874c72fc83ce1239d003a2: 2023-03-27 03:55:09,688 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-27 03:55:09,692 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-27 03:55:09,694 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/9ca06fff42104415a8c39a7223a455af, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/0ea65611c6ce4b80a7fa1a7b13e13710, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/cc6f9313ad64415aba03ab7d342199f9, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/f11c1db0b6a44d3581c825a7593cd87f, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/e2d05847ceb545feabce03099c5d9c30] to archive 2023-03-27 03:55:09,695 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:09,697 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/9ca06fff42104415a8c39a7223a455af to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/9ca06fff42104415a8c39a7223a455af 2023-03-27 03:55:09,697 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-27 03:55:09,698 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/0ea65611c6ce4b80a7fa1a7b13e13710 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/0ea65611c6ce4b80a7fa1a7b13e13710 2023-03-27 03:55:09,698 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-27 03:55:09,699 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/cc6f9313ad64415aba03ab7d342199f9 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/cc6f9313ad64415aba03ab7d342199f9 2023-03-27 03:55:09,700 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/f11c1db0b6a44d3581c825a7593cd87f to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/f11c1db0b6a44d3581c825a7593cd87f 2023-03-27 03:55:09,701 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/e2d05847ceb545feabce03099c5d9c30 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/e2d05847ceb545feabce03099c5d9c30 2023-03-27 03:55:09,704 INFO [Listener at localhost/36131] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-27 03:55:09,738 INFO [Listener at localhost/36131] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-27 03:55:09,742 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303271e65a5e07ebb4872901644d2a7d60eb4_del 2023-03-27 03:55:09,746 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023032776089d93172f41d680bf3b47f5f12fdb_del 2023-03-27 03:55:09,749 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230327a5f6fb5e1d4c4ca2a673355b05d21eb0_del 2023-03-27 03:55:09,749 INFO [Listener at localhost/36131] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-27 03:55:09,749 DEBUG [Listener at localhost/36131] zookeeper.ReadOnlyZKClient(139): Connect 0x6929937b to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:55:09,754 DEBUG [Listener at localhost/36131] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7406c696, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:55:09,754 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230327 for table='testCompactionWithHFileLink' and column='family1' 2023-03-27 03:55:09,764 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230327 for table='testCompactionWithHFileLink' and column='family1' 2023-03-27 03:55:09,764 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230327 for table='testCompactionWithHFileLink' and column='family1' 2023-03-27 03:55:10,245 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303275c09a50088ba4b8994dbc1920671aa86 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303275c09a50088ba4b8994dbc1920671aa86 2023-03-27 03:55:10,246 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230327 table='testCompactionWithHFileLink' column='family1' 2023-03-27 03:55:10,259 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-27 03:55:10,262 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48870, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-27 03:55:10,269 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-27 03:55:10,271 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58276, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-27 03:55:10,280 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230327/family1/64d390b00df141bc85392f5cbe1b670e first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-27 03:55:10,280 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230327/family1/64d390b00df141bc85392f5cbe1b670e}] 2023-03-27 03:55:10,284 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230327/family1/64d390b00df141bc85392f5cbe1b670e for inclusion in 08cac3032353cd1d586bb4d48f4a89b2/family1 2023-03-27 03:55:10,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-27 03:55:10,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last=1 2023-03-27 03:55:10,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for 08cac3032353cd1d586bb4d48f4a89b2: 2023-03-27 03:55:10,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230327/family1/64d390b00df141bc85392f5cbe1b670e to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__r0m7b95url10ol4qk5mickgrqiu01hvq0stski8jci82728amlfg1h4bit5kutjt/family1/64d390b00df141bc85392f5cbe1b670e 2023-03-27 03:55:10,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__r0m7b95url10ol4qk5mickgrqiu01hvq0stski8jci82728amlfg1h4bit5kutjt/family1/64d390b00df141bc85392f5cbe1b670e as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/aef497a5a90c461598e70b6c98962653_SeqId_29_ 2023-03-27 03:55:10,289 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__r0m7b95url10ol4qk5mickgrqiu01hvq0stski8jci82728amlfg1h4bit5kutjt/family1/64d390b00df141bc85392f5cbe1b670e into 08cac3032353cd1d586bb4d48f4a89b2/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/aef497a5a90c461598e70b6c98962653_SeqId_29_ - updating store file list. 2023-03-27 03:55:10,292 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/aef497a5a90c461598e70b6c98962653_SeqId_29_ into 08cac3032353cd1d586bb4d48f4a89b2/family1 2023-03-27 03:55:10,292 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__r0m7b95url10ol4qk5mickgrqiu01hvq0stski8jci82728amlfg1h4bit5kutjt/family1/64d390b00df141bc85392f5cbe1b670e into 08cac3032353cd1d586bb4d48f4a89b2/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/aef497a5a90c461598e70b6c98962653_SeqId_29_) 2023-03-27 03:55:10,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__r0m7b95url10ol4qk5mickgrqiu01hvq0stski8jci82728amlfg1h4bit5kutjt/family1/64d390b00df141bc85392f5cbe1b670e 2023-03-27 03:55:10,294 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230327 table='testCompactionWithHFileLink' column='family1' 2023-03-27 03:55:10,295 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:10,296 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303272e2c3641420440f5aac83f42c82b6a1d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303272e2c3641420440f5aac83f42c82b6a1d 2023-03-27 03:55:10,297 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023032753f50e4eb60b43e48bdca7f0f84c5806 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023032753f50e4eb60b43e48bdca7f0f84c5806 2023-03-27 03:55:10,298 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023032762936666bd29429093aab0b1a3958593 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023032762936666bd29429093aab0b1a3958593 2023-03-27 03:55:10,299 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230327ce396922440a44a2abbfa143e96b7b4b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230327ce396922440a44a2abbfa143e96b7b4b 2023-03-27 03:55:10,299 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-27 03:55:10,299 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230327' 2023-03-27 03:55:10,299 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230327 for table='testCompactionWithHFileLink' and column='family1' 2023-03-27 03:55:10,643 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303270d0996f4680e451ea8f44f4a3e721b85 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303270d0996f4680e451ea8f44f4a3e721b85 2023-03-27 03:55:10,644 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230327 table='testCompactionWithHFileLink' column='family1' 2023-03-27 03:55:10,648 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303275206a990eda9471baa00d677551677fc to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303275206a990eda9471baa00d677551677fc 2023-03-27 03:55:10,649 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230327 table='testCompactionWithHFileLink' column='family1' 2023-03-27 03:55:10,668 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230327/family1/64982b6df6bb424e92c1d0b5659de81e first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-27 03:55:10,669 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230327/family1/64982b6df6bb424e92c1d0b5659de81e}] 2023-03-27 03:55:10,672 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230327/family1/64982b6df6bb424e92c1d0b5659de81e for inclusion in a96b2587b7874c72fc83ce1239d003a2/family1 2023-03-27 03:55:10,677 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230327/family1/819ae0b699c14551b3e6d55a65252b98 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-27 03:55:10,678 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230327/family1/819ae0b699c14551b3e6d55a65252b98}] 2023-03-27 03:55:10,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-27 03:55:10,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-27 03:55:10,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for a96b2587b7874c72fc83ce1239d003a2: 2023-03-27 03:55:10,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230327/family1/64982b6df6bb424e92c1d0b5659de81e to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__brjtb8pgid9f68ii4mu0mqeu38pbb5dh7e07tujc3tit0isfmruusqnnf2fvgl0n/family1/64982b6df6bb424e92c1d0b5659de81e 2023-03-27 03:55:10,680 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230327/family1/819ae0b699c14551b3e6d55a65252b98 for inclusion in faf5e912b9b1c732d5f1903dc1242b64/family1 2023-03-27 03:55:10,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__brjtb8pgid9f68ii4mu0mqeu38pbb5dh7e07tujc3tit0isfmruusqnnf2fvgl0n/family1/64982b6df6bb424e92c1d0b5659de81e as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/93d69d8e36b4432c9c935798a64deb8b_SeqId_29_ 2023-03-27 03:55:10,682 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__brjtb8pgid9f68ii4mu0mqeu38pbb5dh7e07tujc3tit0isfmruusqnnf2fvgl0n/family1/64982b6df6bb424e92c1d0b5659de81e into a96b2587b7874c72fc83ce1239d003a2/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/93d69d8e36b4432c9c935798a64deb8b_SeqId_29_ - updating store file list. 2023-03-27 03:55:10,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-27 03:55:10,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-27 03:55:10,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for faf5e912b9b1c732d5f1903dc1242b64: 2023-03-27 03:55:10,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230327/family1/819ae0b699c14551b3e6d55a65252b98 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__ev2s15omi9i84cdnbgtt2vokkehqi2m8vgoo55vq6nrbdd2e81jvm194qiacpqqt/family1/819ae0b699c14551b3e6d55a65252b98 2023-03-27 03:55:10,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__ev2s15omi9i84cdnbgtt2vokkehqi2m8vgoo55vq6nrbdd2e81jvm194qiacpqqt/family1/819ae0b699c14551b3e6d55a65252b98 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/57dfbdd4b9634393b1f33479f90b9775_SeqId_29_ 2023-03-27 03:55:10,685 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/93d69d8e36b4432c9c935798a64deb8b_SeqId_29_ into a96b2587b7874c72fc83ce1239d003a2/family1 2023-03-27 03:55:10,685 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__brjtb8pgid9f68ii4mu0mqeu38pbb5dh7e07tujc3tit0isfmruusqnnf2fvgl0n/family1/64982b6df6bb424e92c1d0b5659de81e into a96b2587b7874c72fc83ce1239d003a2/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/93d69d8e36b4432c9c935798a64deb8b_SeqId_29_) 2023-03-27 03:55:10,685 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__ev2s15omi9i84cdnbgtt2vokkehqi2m8vgoo55vq6nrbdd2e81jvm194qiacpqqt/family1/819ae0b699c14551b3e6d55a65252b98 into faf5e912b9b1c732d5f1903dc1242b64/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/57dfbdd4b9634393b1f33479f90b9775_SeqId_29_ - updating store file list. 2023-03-27 03:55:10,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__brjtb8pgid9f68ii4mu0mqeu38pbb5dh7e07tujc3tit0isfmruusqnnf2fvgl0n/family1/64982b6df6bb424e92c1d0b5659de81e 2023-03-27 03:55:10,687 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230327 table='testCompactionWithHFileLink' column='family1' 2023-03-27 03:55:10,688 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:10,688 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/57dfbdd4b9634393b1f33479f90b9775_SeqId_29_ into faf5e912b9b1c732d5f1903dc1242b64/family1 2023-03-27 03:55:10,688 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__ev2s15omi9i84cdnbgtt2vokkehqi2m8vgoo55vq6nrbdd2e81jvm194qiacpqqt/family1/819ae0b699c14551b3e6d55a65252b98 into faf5e912b9b1c732d5f1903dc1242b64/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/57dfbdd4b9634393b1f33479f90b9775_SeqId_29_) 2023-03-27 03:55:10,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__ev2s15omi9i84cdnbgtt2vokkehqi2m8vgoo55vq6nrbdd2e81jvm194qiacpqqt/family1/819ae0b699c14551b3e6d55a65252b98 2023-03-27 03:55:10,689 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303276ef5ec9cb35a4d9bb494381e01572d40 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303276ef5ec9cb35a4d9bb494381e01572d40 2023-03-27 03:55:10,690 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032771c38408a9c64734b2b9a0fc42690dc7 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032771c38408a9c64734b2b9a0fc42690dc7 2023-03-27 03:55:10,690 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230327 table='testCompactionWithHFileLink' column='family1' 2023-03-27 03:55:10,691 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:10,691 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032798ddb3a423784c4681291e1eae6554b1 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023032798ddb3a423784c4681291e1eae6554b1 2023-03-27 03:55:10,692 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230327c743126e94f14fd49aa60700919794b4 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230327c743126e94f14fd49aa60700919794b4 2023-03-27 03:55:10,692 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-27 03:55:10,692 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230327' 2023-03-27 03:55:10,692 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230327059e3f0cf9874fe597ffaaa4cf3e8f2c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230327059e3f0cf9874fe597ffaaa4cf3e8f2c 2023-03-27 03:55:10,692 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230327 for table='testCompactionWithHFileLink' and column='family1' 2023-03-27 03:55:10,693 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032725ea9f84ed0e407b960174b9bfdd5af6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032725ea9f84ed0e407b960174b9bfdd5af6 2023-03-27 03:55:10,694 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303274064056ec0ec4d9caf8180840bc868d6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303274064056ec0ec4d9caf8180840bc868d6 2023-03-27 03:55:10,695 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230327a6b65f3b1d404ecc864089d09033c520 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230327a6b65f3b1d404ecc864089d09033c520 2023-03-27 03:55:10,695 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-27 03:55:10,695 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230327' 2023-03-27 03:55:10,695 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230327 for table='testCompactionWithHFileLink' and column='family1' 2023-03-27 03:55:10,695 INFO [Listener at localhost/36131] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-27 03:55:10,695 INFO [Listener at localhost/36131] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-03-27 03:55:10,695 INFO [Listener at localhost/36131] compactions.PartitionedMobCompactor(367): [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303271e65a5e07ebb4872901644d2a7d60eb4_del] 2023-03-27 03:55:10,696 DEBUG [Listener at localhost/36131] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:10,697 DEBUG [Listener at localhost/36131] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303271e65a5e07ebb4872901644d2a7d60eb4_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303271e65a5e07ebb4872901644d2a7d60eb4_del 2023-03-27 03:55:10,697 INFO [Listener at localhost/36131] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-27 03:55:10,697 INFO [Listener at localhost/36131] compactions.PartitionedMobCompactor(367): [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032776089d93172f41d680bf3b47f5f12fdb_del] 2023-03-27 03:55:10,698 DEBUG [Listener at localhost/36131] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:10,699 DEBUG [Listener at localhost/36131] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032776089d93172f41d680bf3b47f5f12fdb_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023032776089d93172f41d680bf3b47f5f12fdb_del 2023-03-27 03:55:10,699 INFO [Listener at localhost/36131] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-27 03:55:10,699 INFO [Listener at localhost/36131] compactions.PartitionedMobCompactor(367): [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230327a5f6fb5e1d4c4ca2a673355b05d21eb0_del] 2023-03-27 03:55:10,699 DEBUG [Listener at localhost/36131] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:10,700 DEBUG [Listener at localhost/36131] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230327a5f6fb5e1d4c4ca2a673355b05d21eb0_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230327a5f6fb5e1d4c4ca2a673355b05d21eb0_del 2023-03-27 03:55:10,700 INFO [Listener at localhost/36131] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-03-27 03:55:10,703 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-27 03:55:10,705 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-27 03:55:10,708 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-27 03:55:10,709 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-27 03:55:10,714 INFO [Listener at localhost/36131] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-03-27 03:55:10,715 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-03-27 03:55:10,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-03-27 03:55:10,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-27 03:55:10,718 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889310717"}]},"ts":"1679889310717"} 2023-03-27 03:55:10,719 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-03-27 03:55:10,720 INFO [PEWorker-2] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-03-27 03:55:10,721 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=08cac3032353cd1d586bb4d48f4a89b2, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=faf5e912b9b1c732d5f1903dc1242b64, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a96b2587b7874c72fc83ce1239d003a2, UNASSIGN}] 2023-03-27 03:55:10,722 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=faf5e912b9b1c732d5f1903dc1242b64, UNASSIGN 2023-03-27 03:55:10,722 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=08cac3032353cd1d586bb4d48f4a89b2, UNASSIGN 2023-03-27 03:55:10,722 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a96b2587b7874c72fc83ce1239d003a2, UNASSIGN 2023-03-27 03:55:10,722 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=faf5e912b9b1c732d5f1903dc1242b64, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:10,722 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=08cac3032353cd1d586bb4d48f4a89b2, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:10,723 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=a96b2587b7874c72fc83ce1239d003a2, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:10,723 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679889310722"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889310722"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889310722"}]},"ts":"1679889310722"} 2023-03-27 03:55:10,723 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679889310722"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889310722"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889310722"}]},"ts":"1679889310722"} 2023-03-27 03:55:10,723 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679889310722"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889310722"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889310722"}]},"ts":"1679889310722"} 2023-03-27 03:55:10,724 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; CloseRegionProcedure faf5e912b9b1c732d5f1903dc1242b64, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:55:10,724 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=89, state=RUNNABLE; CloseRegionProcedure 08cac3032353cd1d586bb4d48f4a89b2, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:55:10,725 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=91, state=RUNNABLE; CloseRegionProcedure a96b2587b7874c72fc83ce1239d003a2, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:55:10,818 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-27 03:55:10,875 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:55:10,875 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing faf5e912b9b1c732d5f1903dc1242b64, disabling compactions & flushes 2023-03-27 03:55:10,875 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:10,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:10,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. after waiting 0 ms 2023-03-27 03:55:10,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:10,879 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-27 03:55:10,880 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:10,880 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for faf5e912b9b1c732d5f1903dc1242b64: 2023-03-27 03:55:10,882 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:55:10,882 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:55:10,882 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a96b2587b7874c72fc83ce1239d003a2, disabling compactions & flushes 2023-03-27 03:55:10,882 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:10,882 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:10,882 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. after waiting 0 ms 2023-03-27 03:55:10,882 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:10,882 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=faf5e912b9b1c732d5f1903dc1242b64, regionState=CLOSED 2023-03-27 03:55:10,882 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679889310882"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889310882"}]},"ts":"1679889310882"} 2023-03-27 03:55:10,884 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-03-27 03:55:10,885 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; CloseRegionProcedure faf5e912b9b1c732d5f1903dc1242b64, server=jenkins-hbase4.apache.org,39727,1679889218560 in 159 msec 2023-03-27 03:55:10,885 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-27 03:55:10,886 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=faf5e912b9b1c732d5f1903dc1242b64, UNASSIGN in 165 msec 2023-03-27 03:55:10,886 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:10,886 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a96b2587b7874c72fc83ce1239d003a2: 2023-03-27 03:55:10,887 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:55:10,887 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:55:10,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 08cac3032353cd1d586bb4d48f4a89b2, disabling compactions & flushes 2023-03-27 03:55:10,887 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:10,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:10,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. after waiting 0 ms 2023-03-27 03:55:10,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:10,887 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=a96b2587b7874c72fc83ce1239d003a2, regionState=CLOSED 2023-03-27 03:55:10,888 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679889310887"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889310887"}]},"ts":"1679889310887"} 2023-03-27 03:55:10,890 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=91 2023-03-27 03:55:10,890 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=91, state=SUCCESS; CloseRegionProcedure a96b2587b7874c72fc83ce1239d003a2, server=jenkins-hbase4.apache.org,39727,1679889218560 in 164 msec 2023-03-27 03:55:10,891 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a96b2587b7874c72fc83ce1239d003a2, UNASSIGN in 170 msec 2023-03-27 03:55:10,891 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-03-27 03:55:10,892 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:10,892 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 08cac3032353cd1d586bb4d48f4a89b2: 2023-03-27 03:55:10,893 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:55:10,893 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=08cac3032353cd1d586bb4d48f4a89b2, regionState=CLOSED 2023-03-27 03:55:10,893 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679889310893"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889310893"}]},"ts":"1679889310893"} 2023-03-27 03:55:10,895 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=89 2023-03-27 03:55:10,895 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=89, state=SUCCESS; CloseRegionProcedure 08cac3032353cd1d586bb4d48f4a89b2, server=jenkins-hbase4.apache.org,39727,1679889218560 in 170 msec 2023-03-27 03:55:10,896 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-03-27 03:55:10,896 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=08cac3032353cd1d586bb4d48f4a89b2, UNASSIGN in 175 msec 2023-03-27 03:55:10,896 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889310896"}]},"ts":"1679889310896"} 2023-03-27 03:55:10,897 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-03-27 03:55:10,899 INFO [PEWorker-5] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-03-27 03:55:10,900 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 184 msec 2023-03-27 03:55:11,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-03-27 03:55:11,019 INFO [Listener at localhost/36131] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-03-27 03:55:11,029 INFO [Listener at localhost/36131] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1679889301486-1679889311029 2023-03-27 03:55:11,030 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1679889301486-1679889311029 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-27 03:55:11,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1679889311030 (current time:1679889311030). 2023-03-27 03:55:11,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-03-27 03:55:11,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] zookeeper.ReadOnlyZKClient(139): Connect 0x73cf7228 to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:55:11,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@144847aa, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:55:11,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-27 03:55:11,038 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48878, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-27 03:55:11,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x73cf7228 to 127.0.0.1:62246 2023-03-27 03:55:11,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:55:11,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-03-27 03:55:11,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1679889301486-1679889311029 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-27 03:55:11,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-27 03:55:11,042 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-27 03:55:11,042 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1679889911042 2023-03-27 03:55:11,042 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1679889301486-1679889311029 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-03-27 03:55:11,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1679889301486-1679889311029 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-27 03:55:11,043 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1679889301486-1679889311029 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:11,043 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-03-27 03:55:11,143 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-27 03:55:11,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1679889301486-1679889311029 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-27 03:55:11,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1679889301486-1679889311029 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-27 03:55:11,144 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-03-27 03:55:11,344 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-27 03:55:11,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1679889301486-1679889311029 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-27 03:55:11,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1679889301486-1679889311029 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-27 03:55:11,345 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-03-27 03:55:11,448 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1679889301486-1679889311029 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-03-27 03:55:11,450 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-27 03:55:11,450 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-27 03:55:11,450 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-27 03:55:11,450 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-27 03:55:11,450 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-27 03:55:11,450 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-03-27 03:55:11,450 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-27 03:55:11,450 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-03-27 03:55:11,453 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/04750adcab514db382f051848edf4589, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/93d69d8e36b4432c9c935798a64deb8b_SeqId_29_] hfiles 2023-03-27 03:55:11,453 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/57dfbdd4b9634393b1f33479f90b9775_SeqId_29_, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/86a5c0a8e88b46c0836ae41a4acee1a1] hfiles 2023-03-27 03:55:11,453 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/04750adcab514db382f051848edf4589 2023-03-27 03:55:11,453 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/57dfbdd4b9634393b1f33479f90b9775_SeqId_29_ 2023-03-27 03:55:11,453 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/93d69d8e36b4432c9c935798a64deb8b_SeqId_29_ 2023-03-27 03:55:11,453 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/86a5c0a8e88b46c0836ae41a4acee1a1 2023-03-27 03:55:11,453 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/aef497a5a90c461598e70b6c98962653_SeqId_29_, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/bdaefea9d97441fa87a50e8f064b4e2b] hfiles 2023-03-27 03:55:11,453 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/aef497a5a90c461598e70b6c98962653_SeqId_29_ 2023-03-27 03:55:11,453 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303275206a990eda9471baa00d677551677fc, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303270d0996f4680e451ea8f44f4a3e721b85, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303275c09a50088ba4b8994dbc1920671aa86] hfiles 2023-03-27 03:55:11,453 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/bdaefea9d97441fa87a50e8f064b4e2b 2023-03-27 03:55:11,453 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303275206a990eda9471baa00d677551677fc 2023-03-27 03:55:11,453 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303270d0996f4680e451ea8f44f4a3e721b85 2023-03-27 03:55:11,453 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303275c09a50088ba4b8994dbc1920671aa86 2023-03-27 03:55:11,453 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/dc2f5b9948654dc2995b26ae720209af] hfiles 2023-03-27 03:55:11,453 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/4cda7786940b4cf58a5177114b209b81] hfiles 2023-03-27 03:55:11,454 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/dc2f5b9948654dc2995b26ae720209af 2023-03-27 03:55:11,454 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/4cda7786940b4cf58a5177114b209b81 2023-03-27 03:55:11,454 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/9111949b017149efbcb1733fd85bbcf3] hfiles 2023-03-27 03:55:11,454 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/9111949b017149efbcb1733fd85bbcf3 2023-03-27 03:55:11,459 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303272376dedadec0473e89d88d6fe66d697a, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230327649d14f722054bc09df576f63c277f36, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023032764acdbba253f4ebc945ddf3c8b1cdd81, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230327c37c951e5b074dd7b21fe060761f0697, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230327fa03c6934b074c4a9b33a6b8521eb183_del, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303271730ef134362450aac84bafaa4706420, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032723d51c3ad3c94363a6fb22c0b5e70cf6, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303272600c42cea424fbf8cf9829ac2850b74_del, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230327ece6da9680f44eda9d86b37189e09673, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230327f55424ed2c6443189f118f16f03c927c, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230327254d12f544e646f1ae34d98ee00fc820, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303275668c83e5a6b43feb6e5d72e713bfa77, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023032791e57ad78612497db69ed50870fd14b6, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230327bba3cebb376f4d6d8e701bac9fe6e212_del, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230327fa6e6618f562408ca885c0890360c478] hfiles 2023-03-27 03:55:11,459 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202303272376dedadec0473e89d88d6fe66d697a 2023-03-27 03:55:11,459 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230327649d14f722054bc09df576f63c277f36 2023-03-27 03:55:11,459 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023032764acdbba253f4ebc945ddf3c8b1cdd81 2023-03-27 03:55:11,459 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230327c37c951e5b074dd7b21fe060761f0697 2023-03-27 03:55:11,459 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230327fa03c6934b074c4a9b33a6b8521eb183_del 2023-03-27 03:55:11,459 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303271730ef134362450aac84bafaa4706420 2023-03-27 03:55:11,459 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023032723d51c3ad3c94363a6fb22c0b5e70cf6 2023-03-27 03:55:11,459 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303272600c42cea424fbf8cf9829ac2850b74_del 2023-03-27 03:55:11,460 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230327ece6da9680f44eda9d86b37189e09673 2023-03-27 03:55:11,460 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230327f55424ed2c6443189f118f16f03c927c 2023-03-27 03:55:11,460 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230327254d12f544e646f1ae34d98ee00fc820 2023-03-27 03:55:11,460 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202303275668c83e5a6b43feb6e5d72e713bfa77 2023-03-27 03:55:11,460 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023032791e57ad78612497db69ed50870fd14b6 2023-03-27 03:55:11,460 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230327bba3cebb376f4d6d8e701bac9fe6e212_del 2023-03-27 03:55:11,460 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230327fa6e6618f562408ca885c0890360c478 2023-03-27 03:55:11,645 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-27 03:55:11,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1679889301486-1679889311029 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-27 03:55:11,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1679889301486-1679889311029 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-27 03:55:11,646 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-03-27 03:55:11,863 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1679889301486-1679889311029 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-03-27 03:55:11,863 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1679889301486-1679889311029 2023-03-27 03:55:11,864 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1679889301486-1679889311029 2023-03-27 03:55:11,888 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1679889301486-1679889311029 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.hbase-snapshot/hbase-failsafe-snaptb-1679889301486-1679889311029 2023-03-27 03:55:12,146 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-27 03:55:12,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1679889301486-1679889311029 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-27 03:55:12,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1679889301486-1679889311029 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-03-27 03:55:12,288 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-27 03:55:12,288 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-03-27 03:55:12,289 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6929937b to 127.0.0.1:62246 2023-03-27 03:55:12,289 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:55:12,289 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-03-27 03:55:12,702 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1679889301486-1679889311029 of table testCompactionWithHFileLink completed 2023-03-27 03:55:12,702 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1679889301486-1679889311029 2023-03-27 03:55:12,703 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1679889301486-1679889311029 2023-03-27 03:55:12,703 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1679889301486-1679889311029 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1679889311042Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1679889301486-1679889311029 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1679889311448 (+406 ms)Consolidate snapshot: hbase-failsafe-snaptb-1679889301486-1679889311029 at 1679889311863 (+415 ms)Loading Region manifests for hbase-failsafe-snaptb-1679889301486-1679889311029 at 1679889311863Writing data manifest for hbase-failsafe-snaptb-1679889301486-1679889311029 at 1679889311866 (+3 ms)Verifying snapshot: hbase-failsafe-snaptb-1679889301486-1679889311029 at 1679889311875 (+9 ms)Snapshot hbase-failsafe-snaptb-1679889301486-1679889311029 of table testCompactionWithHFileLink completed at 1679889312702 (+827 ms) 2023-03-27 03:55:12,704 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-03-27 03:55:12,705 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 1.6630 sec 2023-03-27 03:55:13,289 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-03-27 03:55:13,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1679889301486-1679889311029 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-03-27 03:55:13,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1679889301486-1679889311029 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-03-27 03:55:13,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1679889301486" table: "testCompactionWithHFileLink" creation_time: 1679889301499 type: FLUSH version: 2 ttl: 0 ) 2023-03-27 03:55:13,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1679889301486 as table=testCompactionWithHFileLink 2023-03-27 03:55:13,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-27 03:55:13,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-27 03:55:13,635 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-27 03:55:13,740 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-27 03:55:13,741 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-03-27 03:55:13,741 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-03-27 03:55:13,742 INFO [PEWorker-4] procedure.RestoreSnapshotProcedure(384): Starting restore snapshot={ ss=snaptb-1679889301486 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-03-27 03:55:13,747 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1679889301486" table: "testCompactionWithHFileLink" creation_time: 1679889301499 type: FLUSH version: 2 ttl: 0 2023-03-27 03:55:13,748 DEBUG [PEWorker-4] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink 2023-03-27 03:55:13,753 DEBUG [PEWorker-4] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-03-27 03:55:13,757 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303275206a990eda9471baa00d677551677fc to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303275206a990eda9471baa00d677551677fc 2023-03-27 03:55:13,759 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303270d0996f4680e451ea8f44f4a3e721b85 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303270d0996f4680e451ea8f44f4a3e721b85 2023-03-27 03:55:13,761 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303275c09a50088ba4b8994dbc1920671aa86 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303275c09a50088ba4b8994dbc1920671aa86 2023-03-27 03:55:13,761 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230327059e3f0cf9874fe597ffaaa4cf3e8f2c of snapshot=snaptb-1679889301486 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-27 03:55:13,772 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b2023032725ea9f84ed0e407b960174b9bfdd5af6 of snapshot=snaptb-1679889301486 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-27 03:55:13,778 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202303274064056ec0ec4d9caf8180840bc868d6 of snapshot=snaptb-1679889301486 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-27 03:55:13,782 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230327a6b65f3b1d404ecc864089d09033c520 of snapshot=snaptb-1679889301486 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-27 03:55:13,786 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202303276ef5ec9cb35a4d9bb494381e01572d40 of snapshot=snaptb-1679889301486 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-27 03:55:13,790 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023032771c38408a9c64734b2b9a0fc42690dc7 of snapshot=snaptb-1679889301486 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-27 03:55:13,799 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023032798ddb3a423784c4681291e1eae6554b1 of snapshot=snaptb-1679889301486 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-27 03:55:13,803 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230327c743126e94f14fd49aa60700919794b4 of snapshot=snaptb-1679889301486 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-27 03:55:13,807 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202303272e2c3641420440f5aac83f42c82b6a1d of snapshot=snaptb-1679889301486 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-27 03:55:13,819 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e2023032753f50e4eb60b43e48bdca7f0f84c5806 of snapshot=snaptb-1679889301486 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-27 03:55:13,824 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e2023032762936666bd29429093aab0b1a3958593 of snapshot=snaptb-1679889301486 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-27 03:55:13,829 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230327ce396922440a44a2abbfa143e96b7b4b of snapshot=snaptb-1679889301486 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-03-27 03:55:13,834 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230327fa03c6934b074c4a9b33a6b8521eb183_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230327fa03c6934b074c4a9b33a6b8521eb183_del 2023-03-27 03:55:13,835 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230327bba3cebb376f4d6d8e701bac9fe6e212_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230327bba3cebb376f4d6d8e701bac9fe6e212_del 2023-03-27 03:55:13,837 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303272600c42cea424fbf8cf9829ac2850b74_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202303272600c42cea424fbf8cf9829ac2850b74_del 2023-03-27 03:55:13,837 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(223): region to restore: 08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:55:13,837 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(223): region to restore: a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:55:13,837 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(223): region to restore: faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:55:13,842 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/04750adcab514db382f051848edf4589 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/04750adcab514db382f051848edf4589 2023-03-27 03:55:13,844 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/86a5c0a8e88b46c0836ae41a4acee1a1 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/86a5c0a8e88b46c0836ae41a4acee1a1 2023-03-27 03:55:13,844 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/aef497a5a90c461598e70b6c98962653_SeqId_29_ to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/aef497a5a90c461598e70b6c98962653_SeqId_29_ 2023-03-27 03:55:13,845 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/93d69d8e36b4432c9c935798a64deb8b_SeqId_29_ to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/93d69d8e36b4432c9c935798a64deb8b_SeqId_29_ 2023-03-27 03:55:13,845 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 72ab7fd6b53b42a1ac7eed9e4568659d of snapshot=snaptb-1679889301486 to region=a96b2587b7874c72fc83ce1239d003a2 table=testCompactionWithHFileLink 2023-03-27 03:55:13,845 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/57dfbdd4b9634393b1f33479f90b9775_SeqId_29_ to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/57dfbdd4b9634393b1f33479f90b9775_SeqId_29_ 2023-03-27 03:55:13,845 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 2a10d8329f234324aec49aae55e72329 of snapshot=snaptb-1679889301486 to region=faf5e912b9b1c732d5f1903dc1242b64 table=testCompactionWithHFileLink 2023-03-27 03:55:13,846 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/bdaefea9d97441fa87a50e8f064b4e2b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/bdaefea9d97441fa87a50e8f064b4e2b 2023-03-27 03:55:13,846 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink bdd2fd9286284264bb4230a42ecd2604 of snapshot=snaptb-1679889301486 to region=08cac3032353cd1d586bb4d48f4a89b2 table=testCompactionWithHFileLink 2023-03-27 03:55:13,864 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a02d2e3e10344b54bba8a663ef52c568 of snapshot=snaptb-1679889301486 to region=a96b2587b7874c72fc83ce1239d003a2 table=testCompactionWithHFileLink 2023-03-27 03:55:13,867 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5ae08f2ca4ae401f9e6acd9dfb0f908a of snapshot=snaptb-1679889301486 to region=faf5e912b9b1c732d5f1903dc1242b64 table=testCompactionWithHFileLink 2023-03-27 03:55:13,867 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 64b1111da52f413e9f2a26d0594a1069 of snapshot=snaptb-1679889301486 to region=08cac3032353cd1d586bb4d48f4a89b2 table=testCompactionWithHFileLink 2023-03-27 03:55:13,872 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 750de3b9e4de457880e1f5d6a110e086 of snapshot=snaptb-1679889301486 to region=a96b2587b7874c72fc83ce1239d003a2 table=testCompactionWithHFileLink 2023-03-27 03:55:13,877 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 1ee4d7888f284957b6d9006f42b915a9 of snapshot=snaptb-1679889301486 to region=faf5e912b9b1c732d5f1903dc1242b64 table=testCompactionWithHFileLink 2023-03-27 03:55:13,877 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink dfa9ede98cc54cd9bb80e07c3fc3604c of snapshot=snaptb-1679889301486 to region=a96b2587b7874c72fc83ce1239d003a2 table=testCompactionWithHFileLink 2023-03-27 03:55:13,880 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0de3d9f1ed40478c8ec24614c5d1e5c6 of snapshot=snaptb-1679889301486 to region=faf5e912b9b1c732d5f1903dc1242b64 table=testCompactionWithHFileLink 2023-03-27 03:55:13,885 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/dc2f5b9948654dc2995b26ae720209af to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/dc2f5b9948654dc2995b26ae720209af 2023-03-27 03:55:13,886 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9ca06fff42104415a8c39a7223a455af of snapshot=snaptb-1679889301486 to region=a96b2587b7874c72fc83ce1239d003a2 table=testCompactionWithHFileLink 2023-03-27 03:55:13,896 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5f23b1b075824f629deac284215eaab8 of snapshot=snaptb-1679889301486 to region=08cac3032353cd1d586bb4d48f4a89b2 table=testCompactionWithHFileLink 2023-03-27 03:55:13,900 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0ea65611c6ce4b80a7fa1a7b13e13710 of snapshot=snaptb-1679889301486 to region=a96b2587b7874c72fc83ce1239d003a2 table=testCompactionWithHFileLink 2023-03-27 03:55:13,903 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/4cda7786940b4cf58a5177114b209b81 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/4cda7786940b4cf58a5177114b209b81 2023-03-27 03:55:13,903 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0af7c02db50945349a2e0c8532efb61d of snapshot=snaptb-1679889301486 to region=faf5e912b9b1c732d5f1903dc1242b64 table=testCompactionWithHFileLink 2023-03-27 03:55:13,908 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d867ce5762854f53a4b498bf51357bcd of snapshot=snaptb-1679889301486 to region=08cac3032353cd1d586bb4d48f4a89b2 table=testCompactionWithHFileLink 2023-03-27 03:55:13,909 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink cc6f9313ad64415aba03ab7d342199f9 of snapshot=snaptb-1679889301486 to region=a96b2587b7874c72fc83ce1239d003a2 table=testCompactionWithHFileLink 2023-03-27 03:55:13,919 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 7f0e82322b054d29ae0a7124a90065be of snapshot=snaptb-1679889301486 to region=faf5e912b9b1c732d5f1903dc1242b64 table=testCompactionWithHFileLink 2023-03-27 03:55:13,921 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f11c1db0b6a44d3581c825a7593cd87f of snapshot=snaptb-1679889301486 to region=a96b2587b7874c72fc83ce1239d003a2 table=testCompactionWithHFileLink 2023-03-27 03:55:13,923 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ce6a6eb80bb344489cd1e4201b2aa3ff of snapshot=snaptb-1679889301486 to region=faf5e912b9b1c732d5f1903dc1242b64 table=testCompactionWithHFileLink 2023-03-27 03:55:13,924 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/9111949b017149efbcb1733fd85bbcf3 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/9111949b017149efbcb1733fd85bbcf3 2023-03-27 03:55:13,924 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ba625f70d9564e14ac3e61abdbd37e22 of snapshot=snaptb-1679889301486 to region=08cac3032353cd1d586bb4d48f4a89b2 table=testCompactionWithHFileLink 2023-03-27 03:55:13,931 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 4c5d382fe46d43cdb4bbba094ca5be83 of snapshot=snaptb-1679889301486 to region=08cac3032353cd1d586bb4d48f4a89b2 table=testCompactionWithHFileLink 2023-03-27 03:55:13,931 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink dbf720c6cd314e418def7c5292224d93 of snapshot=snaptb-1679889301486 to region=faf5e912b9b1c732d5f1903dc1242b64 table=testCompactionWithHFileLink 2023-03-27 03:55:13,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-27 03:55:13,944 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 6d4e4ee3a2c44360afc16c2a2ba26705 of snapshot=snaptb-1679889301486 to region=08cac3032353cd1d586bb4d48f4a89b2 table=testCompactionWithHFileLink 2023-03-27 03:55:13,948 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f5f824e90b9d45c7accbb5d4a6a9bd48 of snapshot=snaptb-1679889301486 to region=08cac3032353cd1d586bb4d48f4a89b2 table=testCompactionWithHFileLink 2023-03-27 03:55:13,951 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1679889301486" table: "testCompactionWithHFileLink" creation_time: 1679889301499 type: FLUSH version: 2 ttl: 0 2023-03-27 03:55:13,952 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1679889313952"}]},"ts":"9223372036854775807"} 2023-03-27 03:55:13,952 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1679889313952"}]},"ts":"9223372036854775807"} 2023-03-27 03:55:13,953 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1679889313952"}]},"ts":"9223372036854775807"} 2023-03-27 03:55:13,954 INFO [PEWorker-4] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-03-27 03:55:13,955 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 08cac3032353cd1d586bb4d48f4a89b2, NAME => 'testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => a96b2587b7874c72fc83ce1239d003a2, NAME => 'testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => faf5e912b9b1c732d5f1903dc1242b64, NAME => 'testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64.', STARTKEY => '1', ENDKEY => '2'}] 2023-03-27 03:55:13,955 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679889313953"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889313953"}]},"ts":"1679889313953"} 2023-03-27 03:55:13,955 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679889313953"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889313953"}]},"ts":"1679889313953"} 2023-03-27 03:55:13,955 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679889313953"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889313953"}]},"ts":"1679889313953"} 2023-03-27 03:55:13,956 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-27 03:55:13,956 INFO [PEWorker-4] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-03-27 03:55:13,956 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 08cac3032353cd1d586bb4d48f4a89b2, NAME => 'testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => a96b2587b7874c72fc83ce1239d003a2, NAME => 'testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => faf5e912b9b1c732d5f1903dc1242b64, NAME => 'testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64.', STARTKEY => '1', ENDKEY => '2'}] 2023-03-27 03:55:13,957 INFO [PEWorker-4] procedure.RestoreSnapshotProcedure(457): Restore snapshot={ ss=snaptb-1679889301486 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-03-27 03:55:13,959 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1679889301486" table: "testCompactionWithHFileLink" creation_time: 1679889301499 type: FLUSH version: 2 ttl: 0 ) in 637 msec 2023-03-27 03:55:14,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-03-27 03:55:14,438 INFO [Listener at localhost/36131] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-03-27 03:55:14,438 INFO [Listener at localhost/36131] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1679889301486-1679889311029 2023-03-27 03:55:14,444 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(748): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1679889301486-1679889311029" 2023-03-27 03:55:14,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1679889301486-1679889311029 2023-03-27 03:55:14,449 INFO [Listener at localhost/36131] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-03-27 03:55:14,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$10(2385): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-03-27 03:55:14,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-03-27 03:55:14,462 INFO [PEWorker-1] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-03-27 03:55:14,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-27 03:55:14,462 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889314462"}]},"ts":"1679889314462"} 2023-03-27 03:55:14,463 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-03-27 03:55:14,467 DEBUG [PEWorker-1] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-03-27 03:55:14,467 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=08cac3032353cd1d586bb4d48f4a89b2, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=faf5e912b9b1c732d5f1903dc1242b64, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a96b2587b7874c72fc83ce1239d003a2, ASSIGN}] 2023-03-27 03:55:14,469 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a96b2587b7874c72fc83ce1239d003a2, ASSIGN 2023-03-27 03:55:14,469 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=faf5e912b9b1c732d5f1903dc1242b64, ASSIGN 2023-03-27 03:55:14,469 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=08cac3032353cd1d586bb4d48f4a89b2, ASSIGN 2023-03-27 03:55:14,470 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a96b2587b7874c72fc83ce1239d003a2, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-27 03:55:14,470 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=faf5e912b9b1c732d5f1903dc1242b64, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-27 03:55:14,470 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=08cac3032353cd1d586bb4d48f4a89b2, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-03-27 03:55:14,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-27 03:55:14,622 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=faf5e912b9b1c732d5f1903dc1242b64, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:14,622 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=a96b2587b7874c72fc83ce1239d003a2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:14,622 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679889314622"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889314622"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889314622"}]},"ts":"1679889314622"} 2023-03-27 03:55:14,622 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679889314622"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889314622"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889314622"}]},"ts":"1679889314622"} 2023-03-27 03:55:14,622 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=08cac3032353cd1d586bb4d48f4a89b2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:14,622 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679889314622"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889314622"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889314622"}]},"ts":"1679889314622"} 2023-03-27 03:55:14,624 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=99, state=RUNNABLE; OpenRegionProcedure faf5e912b9b1c732d5f1903dc1242b64, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:55:14,625 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=100, state=RUNNABLE; OpenRegionProcedure a96b2587b7874c72fc83ce1239d003a2, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:55:14,625 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=98, state=RUNNABLE; OpenRegionProcedure 08cac3032353cd1d586bb4d48f4a89b2, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:55:14,764 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-27 03:55:14,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:14,791 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 08cac3032353cd1d586bb4d48f4a89b2, NAME => 'testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2.', STARTKEY => '', ENDKEY => '1'} 2023-03-27 03:55:14,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:55:14,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:55:14,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:55:14,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:55:14,793 INFO [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:55:14,794 DEBUG [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1 2023-03-27 03:55:14,794 DEBUG [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1 2023-03-27 03:55:14,795 INFO [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 08cac3032353cd1d586bb4d48f4a89b2 columnFamilyName family1 2023-03-27 03:55:14,811 DEBUG [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-5f23b1b075824f629deac284215eaab8 2023-03-27 03:55:14,819 DEBUG [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-64b1111da52f413e9f2a26d0594a1069 2023-03-27 03:55:14,824 DEBUG [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-bdd2fd9286284264bb4230a42ecd2604 2023-03-27 03:55:14,832 DEBUG [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-d867ce5762854f53a4b498bf51357bcd 2023-03-27 03:55:14,832 INFO [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] regionserver.HStore(310): Store=08cac3032353cd1d586bb4d48f4a89b2/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:55:14,833 INFO [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:55:14,834 DEBUG [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2 2023-03-27 03:55:14,834 DEBUG [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2 2023-03-27 03:55:14,834 INFO [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 08cac3032353cd1d586bb4d48f4a89b2 columnFamilyName family2 2023-03-27 03:55:14,843 DEBUG [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-4c5d382fe46d43cdb4bbba094ca5be83 2023-03-27 03:55:14,854 DEBUG [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-6d4e4ee3a2c44360afc16c2a2ba26705 2023-03-27 03:55:14,862 DEBUG [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-ba625f70d9564e14ac3e61abdbd37e22 2023-03-27 03:55:14,877 DEBUG [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-f5f824e90b9d45c7accbb5d4a6a9bd48 2023-03-27 03:55:14,878 INFO [StoreOpener-08cac3032353cd1d586bb4d48f4a89b2-1] regionserver.HStore(310): Store=08cac3032353cd1d586bb4d48f4a89b2/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:55:14,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:55:14,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:55:14,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-27 03:55:14,883 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:55:14,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 08cac3032353cd1d586bb4d48f4a89b2; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72836904, jitterRate=0.08535444736480713}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-27 03:55:14,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 08cac3032353cd1d586bb4d48f4a89b2: 2023-03-27 03:55:14,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2., pid=103, masterSystemTime=1679889314788 2023-03-27 03:55:14,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-27 03:55:14,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-27 03:55:14,892 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-27 03:55:14,893 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-27 03:55:14,894 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=08cac3032353cd1d586bb4d48f4a89b2, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:14,895 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679889314894"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889314894"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889314894"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889314894"}]},"ts":"1679889314894"} 2023-03-27 03:55:14,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:14,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:14,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:14,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a96b2587b7874c72fc83ce1239d003a2, NAME => 'testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2.', STARTKEY => '2', ENDKEY => ''} 2023-03-27 03:55:14,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:55:14,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:55:14,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:55:14,897 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-27 03:55:14,897 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:55:14,897 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.HStore(1912): 08cac3032353cd1d586bb4d48f4a89b2/family2 is initiating minor compaction (all files) 2023-03-27 03:55:14,897 INFO [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 08cac3032353cd1d586bb4d48f4a89b2/family2 in testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:14,897 INFO [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-ba625f70d9564e14ac3e61abdbd37e22, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-4c5d382fe46d43cdb4bbba094ca5be83, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-6d4e4ee3a2c44360afc16c2a2ba26705, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-f5f824e90b9d45c7accbb5d4a6a9bd48] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp, totalSize=20.4 K 2023-03-27 03:55:14,897 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-27 03:55:14,897 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1912): 08cac3032353cd1d586bb4d48f4a89b2/family1 is initiating minor compaction (all files) 2023-03-27 03:55:14,898 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 08cac3032353cd1d586bb4d48f4a89b2/family1 in testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:14,898 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-bdd2fd9286284264bb4230a42ecd2604, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-64b1111da52f413e9f2a26d0594a1069, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-5f23b1b075824f629deac284215eaab8, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-d867ce5762854f53a4b498bf51357bcd] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp, totalSize=21.6 K 2023-03-27 03:55:14,899 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=98 2023-03-27 03:55:14,899 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=98, state=SUCCESS; OpenRegionProcedure 08cac3032353cd1d586bb4d48f4a89b2, server=jenkins-hbase4.apache.org,39727,1679889218560 in 272 msec 2023-03-27 03:55:14,900 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=08cac3032353cd1d586bb4d48f4a89b2, ASSIGN in 432 msec 2023-03-27 03:55:14,901 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-ba625f70d9564e14ac3e61abdbd37e22, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889296063 2023-03-27 03:55:14,902 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-bdd2fd9286284264bb4230a42ecd2604, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889296063 2023-03-27 03:55:14,902 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-4c5d382fe46d43cdb4bbba094ca5be83, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889297513 2023-03-27 03:55:14,903 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-64b1111da52f413e9f2a26d0594a1069, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889297513 2023-03-27 03:55:14,904 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-6d4e4ee3a2c44360afc16c2a2ba26705, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889299345 2023-03-27 03:55:14,904 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-5f23b1b075824f629deac284215eaab8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889299345 2023-03-27 03:55:14,905 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-f5f824e90b9d45c7accbb5d4a6a9bd48, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889300817 2023-03-27 03:55:14,905 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-d867ce5762854f53a4b498bf51357bcd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889300817 2023-03-27 03:55:14,916 INFO [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:55:14,924 DEBUG [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1 2023-03-27 03:55:14,925 DEBUG [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1 2023-03-27 03:55:14,925 INFO [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a96b2587b7874c72fc83ce1239d003a2 columnFamilyName family1 2023-03-27 03:55:14,988 DEBUG [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-72ab7fd6b53b42a1ac7eed9e4568659d 2023-03-27 03:55:15,065 DEBUG [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-750de3b9e4de457880e1f5d6a110e086 2023-03-27 03:55:15,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-27 03:55:15,093 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] throttle.PressureAwareThroughputController(145): 08cac3032353cd1d586bb4d48f4a89b2#family1#compaction#114 average throughput is 0.04 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:15,104 INFO [RS:0;jenkins-hbase4:39727-longCompactions-0] throttle.PressureAwareThroughputController(145): 08cac3032353cd1d586bb4d48f4a89b2#family2#compaction#115 average throughput is 0.02 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:15,140 DEBUG [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-a02d2e3e10344b54bba8a663ef52c568 2023-03-27 03:55:15,156 DEBUG [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-dfa9ede98cc54cd9bb80e07c3fc3604c 2023-03-27 03:55:15,157 INFO [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] regionserver.HStore(310): Store=a96b2587b7874c72fc83ce1239d003a2/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:55:15,157 INFO [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:55:15,158 DEBUG [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2 2023-03-27 03:55:15,158 DEBUG [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2 2023-03-27 03:55:15,159 INFO [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a96b2587b7874c72fc83ce1239d003a2 columnFamilyName family2 2023-03-27 03:55:15,182 DEBUG [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-0ea65611c6ce4b80a7fa1a7b13e13710 2023-03-27 03:55:15,188 DEBUG [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-9ca06fff42104415a8c39a7223a455af 2023-03-27 03:55:15,196 DEBUG [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-cc6f9313ad64415aba03ab7d342199f9 2023-03-27 03:55:15,203 DEBUG [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-f11c1db0b6a44d3581c825a7593cd87f 2023-03-27 03:55:15,212 INFO [StoreOpener-a96b2587b7874c72fc83ce1239d003a2-1] regionserver.HStore(310): Store=a96b2587b7874c72fc83ce1239d003a2/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:55:15,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:55:15,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:55:15,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-27 03:55:15,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:55:15,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a96b2587b7874c72fc83ce1239d003a2; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59121684, jitterRate=-0.11901825666427612}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-27 03:55:15,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a96b2587b7874c72fc83ce1239d003a2: 2023-03-27 03:55:15,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2., pid=102, masterSystemTime=1679889314788 2023-03-27 03:55:15,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-27 03:55:15,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-27 03:55:15,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:15,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:15,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:15,224 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=a96b2587b7874c72fc83ce1239d003a2, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:15,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => faf5e912b9b1c732d5f1903dc1242b64, NAME => 'testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64.', STARTKEY => '1', ENDKEY => '2'} 2023-03-27 03:55:15,224 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1679889315224"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889315224"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889315224"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889315224"}]},"ts":"1679889315224"} 2023-03-27 03:55:15,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:55:15,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:55:15,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:55:15,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:55:15,227 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=100 2023-03-27 03:55:15,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=100, state=SUCCESS; OpenRegionProcedure a96b2587b7874c72fc83ce1239d003a2, server=jenkins-hbase4.apache.org,39727,1679889218560 in 601 msec 2023-03-27 03:55:15,229 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=a96b2587b7874c72fc83ce1239d003a2, ASSIGN in 761 msec 2023-03-27 03:55:15,232 INFO [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:55:15,233 DEBUG [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1 2023-03-27 03:55:15,233 DEBUG [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1 2023-03-27 03:55:15,233 INFO [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region faf5e912b9b1c732d5f1903dc1242b64 columnFamilyName family1 2023-03-27 03:55:15,243 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family2/2b5a613bad984c85a126a41f4bd83fdd as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/2b5a613bad984c85a126a41f4bd83fdd 2023-03-27 03:55:15,249 DEBUG [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-0de3d9f1ed40478c8ec24614c5d1e5c6 2023-03-27 03:55:15,250 INFO [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 08cac3032353cd1d586bb4d48f4a89b2/family2 of 08cac3032353cd1d586bb4d48f4a89b2 into 2b5a613bad984c85a126a41f4bd83fdd(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:55:15,251 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 08cac3032353cd1d586bb4d48f4a89b2: 2023-03-27 03:55:15,251 INFO [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2., storeName=08cac3032353cd1d586bb4d48f4a89b2/family2, priority=12, startTime=1679889314891; duration=0sec 2023-03-27 03:55:15,251 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-27 03:55:15,251 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-27 03:55:15,256 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-27 03:55:15,256 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.HStore(1912): a96b2587b7874c72fc83ce1239d003a2/family1 is initiating minor compaction (all files) 2023-03-27 03:55:15,256 INFO [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.HRegion(2259): Starting compaction of a96b2587b7874c72fc83ce1239d003a2/family1 in testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:15,256 INFO [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-72ab7fd6b53b42a1ac7eed9e4568659d, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-a02d2e3e10344b54bba8a663ef52c568, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-750de3b9e4de457880e1f5d6a110e086, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-dfa9ede98cc54cd9bb80e07c3fc3604c] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp, totalSize=21.6 K 2023-03-27 03:55:15,258 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-72ab7fd6b53b42a1ac7eed9e4568659d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889296066 2023-03-27 03:55:15,259 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-a02d2e3e10344b54bba8a663ef52c568, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889297513 2023-03-27 03:55:15,259 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-750de3b9e4de457880e1f5d6a110e086, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889299345 2023-03-27 03:55:15,260 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-dfa9ede98cc54cd9bb80e07c3fc3604c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889300818 2023-03-27 03:55:15,268 DEBUG [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-1ee4d7888f284957b6d9006f42b915a9 2023-03-27 03:55:15,279 DEBUG [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-2a10d8329f234324aec49aae55e72329 2023-03-27 03:55:15,308 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-ba625f70d9564e14ac3e61abdbd37e22, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-4c5d382fe46d43cdb4bbba094ca5be83, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-6d4e4ee3a2c44360afc16c2a2ba26705, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-f5f824e90b9d45c7accbb5d4a6a9bd48] to archive 2023-03-27 03:55:15,315 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:15,321 DEBUG [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-5ae08f2ca4ae401f9e6acd9dfb0f908a 2023-03-27 03:55:15,322 INFO [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] regionserver.HStore(310): Store=faf5e912b9b1c732d5f1903dc1242b64/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:55:15,322 INFO [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:55:15,322 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-ba625f70d9564e14ac3e61abdbd37e22 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-ba625f70d9564e14ac3e61abdbd37e22 2023-03-27 03:55:15,324 DEBUG [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2 2023-03-27 03:55:15,324 DEBUG [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2 2023-03-27 03:55:15,325 INFO [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region faf5e912b9b1c732d5f1903dc1242b64 columnFamilyName family2 2023-03-27 03:55:15,327 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-4c5d382fe46d43cdb4bbba094ca5be83 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-4c5d382fe46d43cdb4bbba094ca5be83 2023-03-27 03:55:15,330 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-6d4e4ee3a2c44360afc16c2a2ba26705 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-6d4e4ee3a2c44360afc16c2a2ba26705 2023-03-27 03:55:15,332 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-f5f824e90b9d45c7accbb5d4a6a9bd48 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family2/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-f5f824e90b9d45c7accbb5d4a6a9bd48 2023-03-27 03:55:15,348 INFO [RS:0;jenkins-hbase4:39727-longCompactions-0] throttle.PressureAwareThroughputController(145): a96b2587b7874c72fc83ce1239d003a2#family1#compaction#116 average throughput is 0.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:15,353 DEBUG [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-0af7c02db50945349a2e0c8532efb61d 2023-03-27 03:55:15,377 DEBUG [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-7f0e82322b054d29ae0a7124a90065be 2023-03-27 03:55:15,384 DEBUG [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-ce6a6eb80bb344489cd1e4201b2aa3ff 2023-03-27 03:55:15,393 DEBUG [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] regionserver.HStore(539): loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-dbf720c6cd314e418def7c5292224d93 2023-03-27 03:55:15,393 INFO [StoreOpener-faf5e912b9b1c732d5f1903dc1242b64-1] regionserver.HStore(310): Store=faf5e912b9b1c732d5f1903dc1242b64/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:55:15,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:55:15,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:55:15,400 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-27 03:55:15,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:55:15,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened faf5e912b9b1c732d5f1903dc1242b64; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64233966, jitterRate=-0.04283931851387024}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-27 03:55:15,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for faf5e912b9b1c732d5f1903dc1242b64: 2023-03-27 03:55:15,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64., pid=101, masterSystemTime=1679889314788 2023-03-27 03:55:15,403 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-27 03:55:15,403 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-27 03:55:15,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:15,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:15,405 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=faf5e912b9b1c732d5f1903dc1242b64, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:15,405 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679889315405"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889315405"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889315405"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889315405"}]},"ts":"1679889315405"} 2023-03-27 03:55:15,411 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=99 2023-03-27 03:55:15,411 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=99, state=SUCCESS; OpenRegionProcedure faf5e912b9b1c732d5f1903dc1242b64, server=jenkins-hbase4.apache.org,39727,1679889218560 in 784 msec 2023-03-27 03:55:15,413 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=97 2023-03-27 03:55:15,413 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=faf5e912b9b1c732d5f1903dc1242b64, ASSIGN in 944 msec 2023-03-27 03:55:15,414 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889315414"}]},"ts":"1679889315414"} 2023-03-27 03:55:15,416 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-03-27 03:55:15,418 INFO [PEWorker-5] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-03-27 03:55:15,419 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 963 msec 2023-03-27 03:55:15,567 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-03-27 03:55:15,567 INFO [Listener at localhost/36131] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-03-27 03:55:15,578 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-27 03:55:15,594 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-27 03:55:15,613 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-27 03:55:15,614 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-27 03:55:15,624 INFO [Listener at localhost/36131] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-27 03:55:15,645 INFO [Listener at localhost/36131] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-03-27 03:55:15,645 INFO [Listener at localhost/36131] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-03-27 03:55:15,646 DEBUG [Listener at localhost/36131] zookeeper.ReadOnlyZKClient(139): Connect 0x0f1f0589 to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:55:15,663 DEBUG [Listener at localhost/36131] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3a243fca, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:55:15,663 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230327 for table='testCompactionWithHFileLink' and column='family1' 2023-03-27 03:55:15,663 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230327 for table='testCompactionWithHFileLink' and column='family1' 2023-03-27 03:55:15,664 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230327 for table='testCompactionWithHFileLink' and column='family1' 2023-03-27 03:55:15,969 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/.tmp/family1/087c4176abc44e708b0baac994e4fdbc as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/087c4176abc44e708b0baac994e4fdbc 2023-03-27 03:55:15,975 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 08cac3032353cd1d586bb4d48f4a89b2/family1 of 08cac3032353cd1d586bb4d48f4a89b2 into 087c4176abc44e708b0baac994e4fdbc(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-27 03:55:15,976 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 08cac3032353cd1d586bb4d48f4a89b2: 2023-03-27 03:55:15,976 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2., storeName=08cac3032353cd1d586bb4d48f4a89b2/family1, priority=12, startTime=1679889314891; duration=1sec 2023-03-27 03:55:15,976 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-03-27 03:55:15,976 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-27 03:55:15,979 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-27 03:55:15,981 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1912): a96b2587b7874c72fc83ce1239d003a2/family2 is initiating minor compaction (all files) 2023-03-27 03:55:15,981 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a96b2587b7874c72fc83ce1239d003a2/family2 in testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:15,981 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-9ca06fff42104415a8c39a7223a455af, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-0ea65611c6ce4b80a7fa1a7b13e13710, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-cc6f9313ad64415aba03ab7d342199f9, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-f11c1db0b6a44d3581c825a7593cd87f] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp, totalSize=20.4 K 2023-03-27 03:55:15,982 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-9ca06fff42104415a8c39a7223a455af, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889296066 2023-03-27 03:55:15,983 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-0ea65611c6ce4b80a7fa1a7b13e13710, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889297513 2023-03-27 03:55:15,983 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-cc6f9313ad64415aba03ab7d342199f9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889299345 2023-03-27 03:55:15,984 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-f11c1db0b6a44d3581c825a7593cd87f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889300818 2023-03-27 03:55:15,994 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-bdd2fd9286284264bb4230a42ecd2604, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-64b1111da52f413e9f2a26d0594a1069, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-5f23b1b075824f629deac284215eaab8, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-d867ce5762854f53a4b498bf51357bcd] to archive 2023-03-27 03:55:15,996 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:15,998 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-bdd2fd9286284264bb4230a42ecd2604 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-bdd2fd9286284264bb4230a42ecd2604 2023-03-27 03:55:16,000 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-64b1111da52f413e9f2a26d0594a1069 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-64b1111da52f413e9f2a26d0594a1069 2023-03-27 03:55:16,002 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-5f23b1b075824f629deac284215eaab8 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-5f23b1b075824f629deac284215eaab8 2023-03-27 03:55:16,005 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-d867ce5762854f53a4b498bf51357bcd to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/testCompactionWithHFileLink=08cac3032353cd1d586bb4d48f4a89b2-d867ce5762854f53a4b498bf51357bcd 2023-03-27 03:55:16,011 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] throttle.PressureAwareThroughputController(145): a96b2587b7874c72fc83ce1239d003a2#family2#compaction#117 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:16,036 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family2/a5a5bf32cee54c3b8100ba446ff94b07 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/a5a5bf32cee54c3b8100ba446ff94b07 2023-03-27 03:55:16,040 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in a96b2587b7874c72fc83ce1239d003a2/family2 of a96b2587b7874c72fc83ce1239d003a2 into a5a5bf32cee54c3b8100ba446ff94b07(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:55:16,040 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a96b2587b7874c72fc83ce1239d003a2: 2023-03-27 03:55:16,040 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2., storeName=a96b2587b7874c72fc83ce1239d003a2/family2, priority=12, startTime=1679889315222; duration=0sec 2023-03-27 03:55:16,040 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-03-27 03:55:16,042 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-27 03:55:16,044 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-27 03:55:16,044 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1912): faf5e912b9b1c732d5f1903dc1242b64/family2 is initiating minor compaction (all files) 2023-03-27 03:55:16,044 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of faf5e912b9b1c732d5f1903dc1242b64/family2 in testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:16,044 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-0af7c02db50945349a2e0c8532efb61d, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-7f0e82322b054d29ae0a7124a90065be, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-ce6a6eb80bb344489cd1e4201b2aa3ff, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-dbf720c6cd314e418def7c5292224d93] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp, totalSize=20.4 K 2023-03-27 03:55:16,045 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-0af7c02db50945349a2e0c8532efb61d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889296065 2023-03-27 03:55:16,045 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-7f0e82322b054d29ae0a7124a90065be, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889297513 2023-03-27 03:55:16,045 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-ce6a6eb80bb344489cd1e4201b2aa3ff, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889299345 2023-03-27 03:55:16,046 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-dbf720c6cd314e418def7c5292224d93, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889300817 2023-03-27 03:55:16,058 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] throttle.PressureAwareThroughputController(145): faf5e912b9b1c732d5f1903dc1242b64#family2#compaction#118 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:16,097 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-9ca06fff42104415a8c39a7223a455af, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-0ea65611c6ce4b80a7fa1a7b13e13710, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-cc6f9313ad64415aba03ab7d342199f9, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-f11c1db0b6a44d3581c825a7593cd87f] to archive 2023-03-27 03:55:16,097 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:16,099 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-9ca06fff42104415a8c39a7223a455af to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-9ca06fff42104415a8c39a7223a455af 2023-03-27 03:55:16,101 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-0ea65611c6ce4b80a7fa1a7b13e13710 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-0ea65611c6ce4b80a7fa1a7b13e13710 2023-03-27 03:55:16,102 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-cc6f9313ad64415aba03ab7d342199f9 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-cc6f9313ad64415aba03ab7d342199f9 2023-03-27 03:55:16,103 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-f11c1db0b6a44d3581c825a7593cd87f to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family2/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-f11c1db0b6a44d3581c825a7593cd87f 2023-03-27 03:55:16,144 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303270d2d990ec94242a19316a987404c1957 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202303270d2d990ec94242a19316a987404c1957 2023-03-27 03:55:16,145 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230327 table='testCompactionWithHFileLink' column='family1' 2023-03-27 03:55:16,146 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303274a128953da4440c1aab17f7a623a48b8 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202303274a128953da4440c1aab17f7a623a48b8 2023-03-27 03:55:16,148 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230327 table='testCompactionWithHFileLink' column='family1' 2023-03-27 03:55:16,153 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303273f2ee13cbb7543dca5d2a678ff4eedeb to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202303273f2ee13cbb7543dca5d2a678ff4eedeb 2023-03-27 03:55:16,154 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230327 table='testCompactionWithHFileLink' column='family1' 2023-03-27 03:55:16,171 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-27 03:55:16,187 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58946, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-27 03:55:16,191 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/.tmp/family1/f6456549b41f48db9445e828306998c4 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/f6456549b41f48db9445e828306998c4 2023-03-27 03:55:16,195 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-27 03:55:16,201 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46602, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-27 03:55:16,202 INFO [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in a96b2587b7874c72fc83ce1239d003a2/family1 of a96b2587b7874c72fc83ce1239d003a2 into f6456549b41f48db9445e828306998c4(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:55:16,202 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for a96b2587b7874c72fc83ce1239d003a2: 2023-03-27 03:55:16,202 INFO [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2., storeName=a96b2587b7874c72fc83ce1239d003a2/family1, priority=12, startTime=1679889315222; duration=0sec 2023-03-27 03:55:16,202 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-03-27 03:55:16,203 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-03-27 03:55:16,215 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-03-27 03:55:16,215 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.HStore(1912): faf5e912b9b1c732d5f1903dc1242b64/family1 is initiating minor compaction (all files) 2023-03-27 03:55:16,215 INFO [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.HRegion(2259): Starting compaction of faf5e912b9b1c732d5f1903dc1242b64/family1 in testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:16,216 INFO [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-2a10d8329f234324aec49aae55e72329, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-5ae08f2ca4ae401f9e6acd9dfb0f908a, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-1ee4d7888f284957b6d9006f42b915a9, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-0de3d9f1ed40478c8ec24614c5d1e5c6] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp, totalSize=21.6 K 2023-03-27 03:55:16,216 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-2a10d8329f234324aec49aae55e72329, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889296065 2023-03-27 03:55:16,218 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-5ae08f2ca4ae401f9e6acd9dfb0f908a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889297513 2023-03-27 03:55:16,220 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-1ee4d7888f284957b6d9006f42b915a9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889299345 2023-03-27 03:55:16,221 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-0de3d9f1ed40478c8ec24614c5d1e5c6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889300817 2023-03-27 03:55:16,237 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230327/family1/194f6f25941b4abab78fd9cef820eed6 first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-03-27 03:55:16,252 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230327/family1/194f6f25941b4abab78fd9cef820eed6}] 2023-03-27 03:55:16,264 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230327/family1/690e1612233042548d9190b88161c3c0 first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-03-27 03:55:16,265 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230327/family1/0a7720b985024e3aae5e73377c1662cc first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-03-27 03:55:16,265 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230327/family1/194f6f25941b4abab78fd9cef820eed6 for inclusion in faf5e912b9b1c732d5f1903dc1242b64/family1 2023-03-27 03:55:16,266 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230327/family1/0a7720b985024e3aae5e73377c1662cc}] 2023-03-27 03:55:16,266 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=32 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230327/family1/690e1612233042548d9190b88161c3c0}] 2023-03-27 03:55:16,273 INFO [RS:0;jenkins-hbase4:39727-longCompactions-0] throttle.PressureAwareThroughputController(145): faf5e912b9b1c732d5f1903dc1242b64#family1#compaction#119 average throughput is 0.14 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:16,273 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230327/family1/690e1612233042548d9190b88161c3c0 for inclusion in 08cac3032353cd1d586bb4d48f4a89b2/family1 2023-03-27 03:55:16,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-03-27 03:55:16,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-27 03:55:16,274 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230327/family1/0a7720b985024e3aae5e73377c1662cc for inclusion in a96b2587b7874c72fc83ce1239d003a2/family1 2023-03-27 03:55:16,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for faf5e912b9b1c732d5f1903dc1242b64: 2023-03-27 03:55:16,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230327/family1/194f6f25941b4abab78fd9cef820eed6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__bivsn3d9heig530a57vrlq7853srvi1u42j7orfnelhjmh4so2i3cf9l6a1te63d/family1/194f6f25941b4abab78fd9cef820eed6 2023-03-27 03:55:16,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-03-27 03:55:16,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last=1 2023-03-27 03:55:16,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for 08cac3032353cd1d586bb4d48f4a89b2: 2023-03-27 03:55:16,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-03-27 03:55:16,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39727] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-27 03:55:16,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for a96b2587b7874c72fc83ce1239d003a2: 2023-03-27 03:55:16,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230327/family1/690e1612233042548d9190b88161c3c0 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__b3aijh4e0q6f4csjgi1so5lbvgd40ljonl4gfjmauuphlva4i4f7tb2rcljp0ebb/family1/690e1612233042548d9190b88161c3c0 2023-03-27 03:55:16,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230327/family1/0a7720b985024e3aae5e73377c1662cc to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__gceplh7mlu3e2sg3dq6f8dvr1cbbu6it1ofcf8amfld2h03q420r0ehsaqq7i40q/family1/0a7720b985024e3aae5e73377c1662cc 2023-03-27 03:55:16,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__bivsn3d9heig530a57vrlq7853srvi1u42j7orfnelhjmh4so2i3cf9l6a1te63d/family1/194f6f25941b4abab78fd9cef820eed6 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/50bd8875ad8445c0b85f6b58a0317060_SeqId_34_ 2023-03-27 03:55:16,283 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__bivsn3d9heig530a57vrlq7853srvi1u42j7orfnelhjmh4so2i3cf9l6a1te63d/family1/194f6f25941b4abab78fd9cef820eed6 into faf5e912b9b1c732d5f1903dc1242b64/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/50bd8875ad8445c0b85f6b58a0317060_SeqId_34_ - updating store file list. 2023-03-27 03:55:16,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__b3aijh4e0q6f4csjgi1so5lbvgd40ljonl4gfjmauuphlva4i4f7tb2rcljp0ebb/family1/690e1612233042548d9190b88161c3c0 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/fec2e5eb26d84bd0b3ad50734890c8bc_SeqId_36_ 2023-03-27 03:55:16,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__gceplh7mlu3e2sg3dq6f8dvr1cbbu6it1ofcf8amfld2h03q420r0ehsaqq7i40q/family1/0a7720b985024e3aae5e73377c1662cc as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/a6164db850ae4f3da06a4bc6cf50db42_SeqId_36_ 2023-03-27 03:55:16,285 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__b3aijh4e0q6f4csjgi1so5lbvgd40ljonl4gfjmauuphlva4i4f7tb2rcljp0ebb/family1/690e1612233042548d9190b88161c3c0 into 08cac3032353cd1d586bb4d48f4a89b2/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/fec2e5eb26d84bd0b3ad50734890c8bc_SeqId_36_ - updating store file list. 2023-03-27 03:55:16,286 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__gceplh7mlu3e2sg3dq6f8dvr1cbbu6it1ofcf8amfld2h03q420r0ehsaqq7i40q/family1/0a7720b985024e3aae5e73377c1662cc into a96b2587b7874c72fc83ce1239d003a2/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/a6164db850ae4f3da06a4bc6cf50db42_SeqId_36_ - updating store file list. 2023-03-27 03:55:16,289 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-72ab7fd6b53b42a1ac7eed9e4568659d, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-a02d2e3e10344b54bba8a663ef52c568, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-750de3b9e4de457880e1f5d6a110e086, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-dfa9ede98cc54cd9bb80e07c3fc3604c] to archive 2023-03-27 03:55:16,296 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:16,300 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/50bd8875ad8445c0b85f6b58a0317060_SeqId_34_ into faf5e912b9b1c732d5f1903dc1242b64/family1 2023-03-27 03:55:16,300 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__bivsn3d9heig530a57vrlq7853srvi1u42j7orfnelhjmh4so2i3cf9l6a1te63d/family1/194f6f25941b4abab78fd9cef820eed6 into faf5e912b9b1c732d5f1903dc1242b64/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/50bd8875ad8445c0b85f6b58a0317060_SeqId_34_) 2023-03-27 03:55:16,301 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__bivsn3d9heig530a57vrlq7853srvi1u42j7orfnelhjmh4so2i3cf9l6a1te63d/family1/194f6f25941b4abab78fd9cef820eed6 2023-03-27 03:55:16,302 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/a6164db850ae4f3da06a4bc6cf50db42_SeqId_36_ into a96b2587b7874c72fc83ce1239d003a2/family1 2023-03-27 03:55:16,302 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__gceplh7mlu3e2sg3dq6f8dvr1cbbu6it1ofcf8amfld2h03q420r0ehsaqq7i40q/family1/0a7720b985024e3aae5e73377c1662cc into a96b2587b7874c72fc83ce1239d003a2/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/a6164db850ae4f3da06a4bc6cf50db42_SeqId_36_) 2023-03-27 03:55:16,303 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/fec2e5eb26d84bd0b3ad50734890c8bc_SeqId_36_ into 08cac3032353cd1d586bb4d48f4a89b2/family1 2023-03-27 03:55:16,303 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__b3aijh4e0q6f4csjgi1so5lbvgd40ljonl4gfjmauuphlva4i4f7tb2rcljp0ebb/family1/690e1612233042548d9190b88161c3c0 into 08cac3032353cd1d586bb4d48f4a89b2/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/family1/fec2e5eb26d84bd0b3ad50734890c8bc_SeqId_36_) 2023-03-27 03:55:16,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__gceplh7mlu3e2sg3dq6f8dvr1cbbu6it1ofcf8amfld2h03q420r0ehsaqq7i40q/family1/0a7720b985024e3aae5e73377c1662cc 2023-03-27 03:55:16,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testCompactionWithHFileLink__b3aijh4e0q6f4csjgi1so5lbvgd40ljonl4gfjmauuphlva4i4f7tb2rcljp0ebb/family1/690e1612233042548d9190b88161c3c0 2023-03-27 03:55:16,305 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230327 table='testCompactionWithHFileLink' column='family1' 2023-03-27 03:55:16,306 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:16,307 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-72ab7fd6b53b42a1ac7eed9e4568659d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-72ab7fd6b53b42a1ac7eed9e4568659d 2023-03-27 03:55:16,308 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230327 table='testCompactionWithHFileLink' column='family1' 2023-03-27 03:55:16,309 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230327 table='testCompactionWithHFileLink' column='family1' 2023-03-27 03:55:16,309 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:16,310 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:16,310 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230327059e3f0cf9874fe597ffaaa4cf3e8f2c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230327059e3f0cf9874fe597ffaaa4cf3e8f2c 2023-03-27 03:55:16,311 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-a02d2e3e10344b54bba8a663ef52c568 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-a02d2e3e10344b54bba8a663ef52c568 2023-03-27 03:55:16,311 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303276ef5ec9cb35a4d9bb494381e01572d40 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202303276ef5ec9cb35a4d9bb494381e01572d40 2023-03-27 03:55:16,312 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-750de3b9e4de457880e1f5d6a110e086 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-750de3b9e4de457880e1f5d6a110e086 2023-03-27 03:55:16,312 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023032725ea9f84ed0e407b960174b9bfdd5af6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023032725ea9f84ed0e407b960174b9bfdd5af6 2023-03-27 03:55:16,312 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303272e2c3641420440f5aac83f42c82b6a1d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202303272e2c3641420440f5aac83f42c82b6a1d 2023-03-27 03:55:16,313 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-dfa9ede98cc54cd9bb80e07c3fc3604c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/family1/testCompactionWithHFileLink=a96b2587b7874c72fc83ce1239d003a2-dfa9ede98cc54cd9bb80e07c3fc3604c 2023-03-27 03:55:16,313 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023032771c38408a9c64734b2b9a0fc42690dc7 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023032771c38408a9c64734b2b9a0fc42690dc7 2023-03-27 03:55:16,313 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023032753f50e4eb60b43e48bdca7f0f84c5806 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023032753f50e4eb60b43e48bdca7f0f84c5806 2023-03-27 03:55:16,313 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303274064056ec0ec4d9caf8180840bc868d6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202303274064056ec0ec4d9caf8180840bc868d6 2023-03-27 03:55:16,314 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023032798ddb3a423784c4681291e1eae6554b1 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023032798ddb3a423784c4681291e1eae6554b1 2023-03-27 03:55:16,314 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023032762936666bd29429093aab0b1a3958593 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e2023032762936666bd29429093aab0b1a3958593 2023-03-27 03:55:16,315 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230327c743126e94f14fd49aa60700919794b4 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230327c743126e94f14fd49aa60700919794b4 2023-03-27 03:55:16,315 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-27 03:55:16,315 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230327' 2023-03-27 03:55:16,316 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230327 for table='testCompactionWithHFileLink' and column='family1' 2023-03-27 03:55:16,317 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230327a6b65f3b1d404ecc864089d09033c520 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230327a6b65f3b1d404ecc864089d09033c520 2023-03-27 03:55:16,317 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-27 03:55:16,317 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230327ce396922440a44a2abbfa143e96b7b4b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230327ce396922440a44a2abbfa143e96b7b4b 2023-03-27 03:55:16,317 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230327' 2023-03-27 03:55:16,317 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-03-27 03:55:16,317 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230327' 2023-03-27 03:55:16,318 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230327 for table='testCompactionWithHFileLink' and column='family1' 2023-03-27 03:55:16,318 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230327 for table='testCompactionWithHFileLink' and column='family1' 2023-03-27 03:55:16,318 INFO [Listener at localhost/36131] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-03-27 03:55:16,322 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-27 03:55:16,324 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-27 03:55:16,338 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-27 03:55:16,340 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-27 03:55:16,352 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-27 03:55:16,353 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-27 03:55:16,375 INFO [Listener at localhost/36131] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=275 (was 253) - Thread LEAK? -, OpenFileDescriptor=711 (was 630) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=390 (was 348) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=8640 (was 8784) 2023-03-27 03:55:16,391 INFO [Listener at localhost/36131] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=275, OpenFileDescriptor=711, MaxFileDescriptor=60000, SystemLoadAverage=390, ProcessCount=169, AvailableMemoryMB=8639 2023-03-27 03:55:16,397 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$15(3014): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-03-27 03:55:16,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-03-27 03:55:16,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-27 03:55:16,407 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-03-27 03:55:16,409 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 9 msec 2023-03-27 03:55:16,430 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-27 03:55:16,478 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family2/3546e9112d19401b9de0f63541ef8dc0 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/3546e9112d19401b9de0f63541ef8dc0 2023-03-27 03:55:16,485 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in faf5e912b9b1c732d5f1903dc1242b64/family2 of faf5e912b9b1c732d5f1903dc1242b64 into 3546e9112d19401b9de0f63541ef8dc0(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:55:16,485 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for faf5e912b9b1c732d5f1903dc1242b64: 2023-03-27 03:55:16,485 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64., storeName=faf5e912b9b1c732d5f1903dc1242b64/family2, priority=12, startTime=1679889315403; duration=0sec 2023-03-27 03:55:16,485 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:55:16,492 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-0af7c02db50945349a2e0c8532efb61d, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-7f0e82322b054d29ae0a7124a90065be, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-ce6a6eb80bb344489cd1e4201b2aa3ff, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-dbf720c6cd314e418def7c5292224d93] to archive 2023-03-27 03:55:16,493 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:16,494 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-0af7c02db50945349a2e0c8532efb61d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-0af7c02db50945349a2e0c8532efb61d 2023-03-27 03:55:16,495 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-7f0e82322b054d29ae0a7124a90065be to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-7f0e82322b054d29ae0a7124a90065be 2023-03-27 03:55:16,497 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-ce6a6eb80bb344489cd1e4201b2aa3ff to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-ce6a6eb80bb344489cd1e4201b2aa3ff 2023-03-27 03:55:16,498 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-dbf720c6cd314e418def7c5292224d93 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family2/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-dbf720c6cd314e418def7c5292224d93 2023-03-27 03:55:16,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-03-27 03:55:16,508 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-27 03:55:16,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-03-27 03:55:16,510 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-03-27 03:55:16,511 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-03-27 03:55:16,511 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-27 03:55:16,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-27 03:55:16,513 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929 2023-03-27 03:55:16,513 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22 2023-03-27 03:55:16,513 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8 2023-03-27 03:55:16,514 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929 empty. 2023-03-27 03:55:16,514 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8 empty. 2023-03-27 03:55:16,514 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22 empty. 2023-03-27 03:55:16,514 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929 2023-03-27 03:55:16,515 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22 2023-03-27 03:55:16,515 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8 2023-03-27 03:55:16,515 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-03-27 03:55:16,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-27 03:55:16,710 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/.tmp/family1/a90a2dbad568466d97e673e89989a3e8 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/a90a2dbad568466d97e673e89989a3e8 2023-03-27 03:55:16,714 INFO [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in faf5e912b9b1c732d5f1903dc1242b64/family1 of faf5e912b9b1c732d5f1903dc1242b64 into a90a2dbad568466d97e673e89989a3e8(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:55:16,714 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for faf5e912b9b1c732d5f1903dc1242b64: 2023-03-27 03:55:16,714 INFO [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64., storeName=faf5e912b9b1c732d5f1903dc1242b64/family1, priority=12, startTime=1679889315403; duration=0sec 2023-03-27 03:55:16,714 DEBUG [RS:0;jenkins-hbase4:39727-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:55:16,789 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-2a10d8329f234324aec49aae55e72329, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-5ae08f2ca4ae401f9e6acd9dfb0f908a, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-1ee4d7888f284957b6d9006f42b915a9, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-0de3d9f1ed40478c8ec24614c5d1e5c6] to archive 2023-03-27 03:55:16,790 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:16,792 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-2a10d8329f234324aec49aae55e72329 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-2a10d8329f234324aec49aae55e72329 2023-03-27 03:55:16,793 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-5ae08f2ca4ae401f9e6acd9dfb0f908a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-5ae08f2ca4ae401f9e6acd9dfb0f908a 2023-03-27 03:55:16,794 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-1ee4d7888f284957b6d9006f42b915a9 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-1ee4d7888f284957b6d9006f42b915a9 2023-03-27 03:55:16,795 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-0de3d9f1ed40478c8ec24614c5d1e5c6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/family1/testCompactionWithHFileLink=faf5e912b9b1c732d5f1903dc1242b64-0de3d9f1ed40478c8ec24614c5d1e5c6 2023-03-27 03:55:16,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-27 03:55:16,928 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-03-27 03:55:16,930 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7675): creating {ENCODED => 72b74f7fa0bbf3322278037ab0d1d7f8, NAME => 'ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp 2023-03-27 03:55:16,930 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7675): creating {ENCODED => 9ce98af924f76608e57db6d1e9c69a22, NAME => 'ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp 2023-03-27 03:55:16,930 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 7bab4d71becda434aedcb5da6cc18929, NAME => 'ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp 2023-03-27 03:55:16,946 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:55:16,946 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1604): Closing 7bab4d71becda434aedcb5da6cc18929, disabling compactions & flushes 2023-03-27 03:55:16,946 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:16,946 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:16,946 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. after waiting 0 ms 2023-03-27 03:55:16,946 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:16,946 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:16,946 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 7bab4d71becda434aedcb5da6cc18929: 2023-03-27 03:55:17,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-27 03:55:17,339 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:55:17,339 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1604): Closing 72b74f7fa0bbf3322278037ab0d1d7f8, disabling compactions & flushes 2023-03-27 03:55:17,340 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:17,340 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:17,340 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. after waiting 0 ms 2023-03-27 03:55:17,340 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:17,340 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:17,340 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1558): Region close journal for 72b74f7fa0bbf3322278037ab0d1d7f8: 2023-03-27 03:55:17,345 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:55:17,345 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1604): Closing 9ce98af924f76608e57db6d1e9c69a22, disabling compactions & flushes 2023-03-27 03:55:17,345 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:17,345 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:17,345 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. after waiting 0 ms 2023-03-27 03:55:17,345 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:17,345 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:17,345 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1558): Region close journal for 9ce98af924f76608e57db6d1e9c69a22: 2023-03-27 03:55:17,347 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-03-27 03:55:17,348 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679889317348"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889317348"}]},"ts":"1679889317348"} 2023-03-27 03:55:17,348 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679889317348"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889317348"}]},"ts":"1679889317348"} 2023-03-27 03:55:17,349 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1679889317348"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889317348"}]},"ts":"1679889317348"} 2023-03-27 03:55:17,351 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-27 03:55:17,351 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-27 03:55:17,352 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889317351"}]},"ts":"1679889317351"} 2023-03-27 03:55:17,354 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-03-27 03:55:17,357 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=7bab4d71becda434aedcb5da6cc18929, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9ce98af924f76608e57db6d1e9c69a22, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=72b74f7fa0bbf3322278037ab0d1d7f8, ASSIGN}] 2023-03-27 03:55:17,359 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=72b74f7fa0bbf3322278037ab0d1d7f8, ASSIGN 2023-03-27 03:55:17,359 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=7bab4d71becda434aedcb5da6cc18929, ASSIGN 2023-03-27 03:55:17,359 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9ce98af924f76608e57db6d1e9c69a22, ASSIGN 2023-03-27 03:55:17,359 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=72b74f7fa0bbf3322278037ab0d1d7f8, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=false 2023-03-27 03:55:17,360 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=7bab4d71becda434aedcb5da6cc18929, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=false 2023-03-27 03:55:17,360 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9ce98af924f76608e57db6d1e9c69a22, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=false 2023-03-27 03:55:17,512 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=7bab4d71becda434aedcb5da6cc18929, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:17,512 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=72b74f7fa0bbf3322278037ab0d1d7f8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:17,512 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=9ce98af924f76608e57db6d1e9c69a22, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:17,512 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679889317512"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889317512"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889317512"}]},"ts":"1679889317512"} 2023-03-27 03:55:17,512 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1679889317512"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889317512"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889317512"}]},"ts":"1679889317512"} 2023-03-27 03:55:17,512 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679889317512"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889317512"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889317512"}]},"ts":"1679889317512"} 2023-03-27 03:55:17,514 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=107, state=RUNNABLE; OpenRegionProcedure 9ce98af924f76608e57db6d1e9c69a22, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:55:17,514 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=106, state=RUNNABLE; OpenRegionProcedure 7bab4d71becda434aedcb5da6cc18929, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:55:17,515 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=108, state=RUNNABLE; OpenRegionProcedure 72b74f7fa0bbf3322278037ab0d1d7f8, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:55:17,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-27 03:55:17,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:17,668 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7bab4d71becda434aedcb5da6cc18929, NAME => 'ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929.', STARTKEY => '', ENDKEY => '1'} 2023-03-27 03:55:17,668 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 7bab4d71becda434aedcb5da6cc18929 2023-03-27 03:55:17,668 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:55:17,668 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7bab4d71becda434aedcb5da6cc18929 2023-03-27 03:55:17,668 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7bab4d71becda434aedcb5da6cc18929 2023-03-27 03:55:17,672 INFO [StoreOpener-7bab4d71becda434aedcb5da6cc18929-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 7bab4d71becda434aedcb5da6cc18929 2023-03-27 03:55:17,673 DEBUG [StoreOpener-7bab4d71becda434aedcb5da6cc18929-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1 2023-03-27 03:55:17,673 DEBUG [StoreOpener-7bab4d71becda434aedcb5da6cc18929-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1 2023-03-27 03:55:17,673 INFO [StoreOpener-7bab4d71becda434aedcb5da6cc18929-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7bab4d71becda434aedcb5da6cc18929 columnFamilyName family1 2023-03-27 03:55:17,675 INFO [StoreOpener-7bab4d71becda434aedcb5da6cc18929-1] regionserver.HStore(310): Store=7bab4d71becda434aedcb5da6cc18929/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:55:17,675 INFO [StoreOpener-7bab4d71becda434aedcb5da6cc18929-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 7bab4d71becda434aedcb5da6cc18929 2023-03-27 03:55:17,676 DEBUG [StoreOpener-7bab4d71becda434aedcb5da6cc18929-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2 2023-03-27 03:55:17,676 DEBUG [StoreOpener-7bab4d71becda434aedcb5da6cc18929-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2 2023-03-27 03:55:17,677 INFO [StoreOpener-7bab4d71becda434aedcb5da6cc18929-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7bab4d71becda434aedcb5da6cc18929 columnFamilyName family2 2023-03-27 03:55:17,678 INFO [StoreOpener-7bab4d71becda434aedcb5da6cc18929-1] regionserver.HStore(310): Store=7bab4d71becda434aedcb5da6cc18929/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:55:17,679 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929 2023-03-27 03:55:17,682 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929 2023-03-27 03:55:17,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-27 03:55:17,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7bab4d71becda434aedcb5da6cc18929 2023-03-27 03:55:17,687 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:55:17,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7bab4d71becda434aedcb5da6cc18929; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64846187, jitterRate=-0.03371651470661163}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-27 03:55:17,687 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7bab4d71becda434aedcb5da6cc18929: 2023-03-27 03:55:17,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929., pid=110, masterSystemTime=1679889317665 2023-03-27 03:55:17,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:17,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:17,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:17,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 72b74f7fa0bbf3322278037ab0d1d7f8, NAME => 'ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8.', STARTKEY => '2', ENDKEY => ''} 2023-03-27 03:55:17,690 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=7bab4d71becda434aedcb5da6cc18929, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:17,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 72b74f7fa0bbf3322278037ab0d1d7f8 2023-03-27 03:55:17,690 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679889317690"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889317690"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889317690"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889317690"}]},"ts":"1679889317690"} 2023-03-27 03:55:17,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:55:17,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 72b74f7fa0bbf3322278037ab0d1d7f8 2023-03-27 03:55:17,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 72b74f7fa0bbf3322278037ab0d1d7f8 2023-03-27 03:55:17,691 INFO [StoreOpener-72b74f7fa0bbf3322278037ab0d1d7f8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 72b74f7fa0bbf3322278037ab0d1d7f8 2023-03-27 03:55:17,693 DEBUG [StoreOpener-72b74f7fa0bbf3322278037ab0d1d7f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1 2023-03-27 03:55:17,693 DEBUG [StoreOpener-72b74f7fa0bbf3322278037ab0d1d7f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1 2023-03-27 03:55:17,693 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=106 2023-03-27 03:55:17,693 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=106, state=SUCCESS; OpenRegionProcedure 7bab4d71becda434aedcb5da6cc18929, server=jenkins-hbase4.apache.org,39727,1679889218560 in 177 msec 2023-03-27 03:55:17,693 INFO [StoreOpener-72b74f7fa0bbf3322278037ab0d1d7f8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 72b74f7fa0bbf3322278037ab0d1d7f8 columnFamilyName family1 2023-03-27 03:55:17,694 INFO [StoreOpener-72b74f7fa0bbf3322278037ab0d1d7f8-1] regionserver.HStore(310): Store=72b74f7fa0bbf3322278037ab0d1d7f8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:55:17,694 INFO [StoreOpener-72b74f7fa0bbf3322278037ab0d1d7f8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 72b74f7fa0bbf3322278037ab0d1d7f8 2023-03-27 03:55:17,694 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=7bab4d71becda434aedcb5da6cc18929, ASSIGN in 336 msec 2023-03-27 03:55:17,695 DEBUG [StoreOpener-72b74f7fa0bbf3322278037ab0d1d7f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2 2023-03-27 03:55:17,695 DEBUG [StoreOpener-72b74f7fa0bbf3322278037ab0d1d7f8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2 2023-03-27 03:55:17,696 INFO [StoreOpener-72b74f7fa0bbf3322278037ab0d1d7f8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 72b74f7fa0bbf3322278037ab0d1d7f8 columnFamilyName family2 2023-03-27 03:55:17,699 INFO [StoreOpener-72b74f7fa0bbf3322278037ab0d1d7f8-1] regionserver.HStore(310): Store=72b74f7fa0bbf3322278037ab0d1d7f8/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:55:17,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8 2023-03-27 03:55:17,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8 2023-03-27 03:55:17,702 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-27 03:55:17,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 72b74f7fa0bbf3322278037ab0d1d7f8 2023-03-27 03:55:17,704 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:55:17,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 72b74f7fa0bbf3322278037ab0d1d7f8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70878220, jitterRate=0.056167781352996826}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-27 03:55:17,705 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 72b74f7fa0bbf3322278037ab0d1d7f8: 2023-03-27 03:55:17,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8., pid=111, masterSystemTime=1679889317665 2023-03-27 03:55:17,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:17,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:17,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:17,707 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=72b74f7fa0bbf3322278037ab0d1d7f8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:17,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9ce98af924f76608e57db6d1e9c69a22, NAME => 'ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22.', STARTKEY => '1', ENDKEY => '2'} 2023-03-27 03:55:17,707 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1679889317707"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889317707"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889317707"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889317707"}]},"ts":"1679889317707"} 2023-03-27 03:55:17,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 9ce98af924f76608e57db6d1e9c69a22 2023-03-27 03:55:17,708 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:55:17,708 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 9ce98af924f76608e57db6d1e9c69a22 2023-03-27 03:55:17,708 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 9ce98af924f76608e57db6d1e9c69a22 2023-03-27 03:55:17,710 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=108 2023-03-27 03:55:17,710 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=108, state=SUCCESS; OpenRegionProcedure 72b74f7fa0bbf3322278037ab0d1d7f8, server=jenkins-hbase4.apache.org,39727,1679889218560 in 194 msec 2023-03-27 03:55:17,711 INFO [StoreOpener-9ce98af924f76608e57db6d1e9c69a22-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9ce98af924f76608e57db6d1e9c69a22 2023-03-27 03:55:17,711 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=72b74f7fa0bbf3322278037ab0d1d7f8, ASSIGN in 353 msec 2023-03-27 03:55:17,712 DEBUG [StoreOpener-9ce98af924f76608e57db6d1e9c69a22-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1 2023-03-27 03:55:17,712 DEBUG [StoreOpener-9ce98af924f76608e57db6d1e9c69a22-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1 2023-03-27 03:55:17,712 INFO [StoreOpener-9ce98af924f76608e57db6d1e9c69a22-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9ce98af924f76608e57db6d1e9c69a22 columnFamilyName family1 2023-03-27 03:55:17,713 INFO [StoreOpener-9ce98af924f76608e57db6d1e9c69a22-1] regionserver.HStore(310): Store=9ce98af924f76608e57db6d1e9c69a22/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:55:17,713 INFO [StoreOpener-9ce98af924f76608e57db6d1e9c69a22-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 9ce98af924f76608e57db6d1e9c69a22 2023-03-27 03:55:17,714 DEBUG [StoreOpener-9ce98af924f76608e57db6d1e9c69a22-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2 2023-03-27 03:55:17,714 DEBUG [StoreOpener-9ce98af924f76608e57db6d1e9c69a22-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2 2023-03-27 03:55:17,715 INFO [StoreOpener-9ce98af924f76608e57db6d1e9c69a22-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9ce98af924f76608e57db6d1e9c69a22 columnFamilyName family2 2023-03-27 03:55:17,715 INFO [StoreOpener-9ce98af924f76608e57db6d1e9c69a22-1] regionserver.HStore(310): Store=9ce98af924f76608e57db6d1e9c69a22/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:55:17,716 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22 2023-03-27 03:55:17,717 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22 2023-03-27 03:55:17,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-27 03:55:17,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 9ce98af924f76608e57db6d1e9c69a22 2023-03-27 03:55:17,720 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:55:17,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 9ce98af924f76608e57db6d1e9c69a22; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71752183, jitterRate=0.0691908448934555}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-27 03:55:17,721 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 9ce98af924f76608e57db6d1e9c69a22: 2023-03-27 03:55:17,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22., pid=109, masterSystemTime=1679889317665 2023-03-27 03:55:17,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:17,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:17,723 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=9ce98af924f76608e57db6d1e9c69a22, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:17,723 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1679889317723"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889317723"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889317723"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889317723"}]},"ts":"1679889317723"} 2023-03-27 03:55:17,726 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=107 2023-03-27 03:55:17,726 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=107, state=SUCCESS; OpenRegionProcedure 9ce98af924f76608e57db6d1e9c69a22, server=jenkins-hbase4.apache.org,39727,1679889218560 in 210 msec 2023-03-27 03:55:17,727 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=105 2023-03-27 03:55:17,727 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9ce98af924f76608e57db6d1e9c69a22, ASSIGN in 369 msec 2023-03-27 03:55:17,728 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-27 03:55:17,728 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889317728"}]},"ts":"1679889317728"} 2023-03-27 03:55:17,730 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-03-27 03:55:17,733 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-03-27 03:55:17,735 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.2250 sec 2023-03-27 03:55:18,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-03-27 03:55:18,617 INFO [Listener at localhost/36131] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-03-27 03:55:18,624 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:18,625 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:18,626 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:18,636 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:18,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-27 03:55:18,641 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-27 03:55:18,641 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:18,642 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-27 03:55:18,642 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-27 03:55:18,642 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:18,642 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:18,645 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:18,645 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:18,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:18,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:18,645 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:18,645 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:18,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:18,645 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:18,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:18,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:18,646 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-27 03:55:18,650 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-27 03:55:18,650 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-27 03:55:18,650 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:18,650 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-27 03:55:18,652 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:18,652 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. started... 2023-03-27 03:55:18,655 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7bab4d71becda434aedcb5da6cc18929 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-27 03:55:18,656 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:18,656 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. started... 2023-03-27 03:55:18,657 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 9ce98af924f76608e57db6d1e9c69a22 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-27 03:55:18,664 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-27 03:55:18,664 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:18,664 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:18,664 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. started... 2023-03-27 03:55:18,668 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 72b74f7fa0bbf3322278037ab0d1d7f8 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-27 03:55:19,109 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303274280863728994b84bd4be9e6d467f8e7 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303274280863728994b84bd4be9e6d467f8e7 2023-03-27 03:55:19,110 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230327111fe37acbdb48668789203989e5883a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230327111fe37acbdb48668789203989e5883a 2023-03-27 03:55:19,136 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230327a42afebf3aba43fbac870427faff1b88 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230327a42afebf3aba43fbac870427faff1b88 2023-03-27 03:55:19,142 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family1/f29e92c205ff4ec4a1aa35defdb9c29c 2023-03-27 03:55:19,515 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family1/63d80d62e7cc4f1598413736a7da0d92 2023-03-27 03:55:19,516 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family1/0d8e89097f6e4d68a18a2dbd04ed6dfb 2023-03-27 03:55:19,549 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303278d6457573b404b2788f1155a1e71b50f to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202303278d6457573b404b2788f1155a1e71b50f 2023-03-27 03:55:19,550 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230327bc8ac4c983534c098b6fc08d88d1d1b0 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230327bc8ac4c983534c098b6fc08d88d1d1b0 2023-03-27 03:55:19,555 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family2/4a249aca31574b1f8cc171212769055b 2023-03-27 03:55:19,560 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032703735b89f21649bb83c4e5b71309192d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023032703735b89f21649bb83c4e5b71309192d 2023-03-27 03:55:19,562 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family1/0d8e89097f6e4d68a18a2dbd04ed6dfb as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/0d8e89097f6e4d68a18a2dbd04ed6dfb 2023-03-27 03:55:19,567 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/0d8e89097f6e4d68a18a2dbd04ed6dfb, entries=4, sequenceid=5, filesize=5.4 K 2023-03-27 03:55:19,569 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family2/4a249aca31574b1f8cc171212769055b as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/4a249aca31574b1f8cc171212769055b 2023-03-27 03:55:19,573 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family2/2e74877b929c49099c6ed1b42da58dac 2023-03-27 03:55:19,578 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/4a249aca31574b1f8cc171212769055b, entries=2, sequenceid=5, filesize=5.1 K 2023-03-27 03:55:19,579 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 7bab4d71becda434aedcb5da6cc18929 in 924ms, sequenceid=5, compaction requested=false 2023-03-27 03:55:19,579 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-03-27 03:55:19,580 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7bab4d71becda434aedcb5da6cc18929: 2023-03-27 03:55:19,580 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:19,580 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-27 03:55:19,581 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family2/7f4b087b465f43aa8632446e7c860dbe 2023-03-27 03:55:19,583 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family1/f29e92c205ff4ec4a1aa35defdb9c29c as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/f29e92c205ff4ec4a1aa35defdb9c29c 2023-03-27 03:55:19,594 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family1/63d80d62e7cc4f1598413736a7da0d92 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/63d80d62e7cc4f1598413736a7da0d92 2023-03-27 03:55:19,597 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/f29e92c205ff4ec4a1aa35defdb9c29c, entries=4, sequenceid=5, filesize=5.4 K 2023-03-27 03:55:19,599 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family2/2e74877b929c49099c6ed1b42da58dac as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/2e74877b929c49099c6ed1b42da58dac 2023-03-27 03:55:19,602 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/63d80d62e7cc4f1598413736a7da0d92, entries=4, sequenceid=5, filesize=5.4 K 2023-03-27 03:55:19,602 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family2/7f4b087b465f43aa8632446e7c860dbe as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/7f4b087b465f43aa8632446e7c860dbe 2023-03-27 03:55:19,605 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/2e74877b929c49099c6ed1b42da58dac, entries=2, sequenceid=5, filesize=5.1 K 2023-03-27 03:55:19,606 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 72b74f7fa0bbf3322278037ab0d1d7f8 in 938ms, sequenceid=5, compaction requested=false 2023-03-27 03:55:19,607 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 72b74f7fa0bbf3322278037ab0d1d7f8: 2023-03-27 03:55:19,608 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:19,608 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-27 03:55:19,608 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/7f4b087b465f43aa8632446e7c860dbe, entries=2, sequenceid=5, filesize=5.1 K 2023-03-27 03:55:19,610 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 9ce98af924f76608e57db6d1e9c69a22 in 953ms, sequenceid=5, compaction requested=false 2023-03-27 03:55:19,610 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 9ce98af924f76608e57db6d1e9c69a22: 2023-03-27 03:55:19,610 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:19,610 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-27 03:55:19,610 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-27 03:55:19,610 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:19,610 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-27 03:55:19,610 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-27 03:55:19,612 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:19,612 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:19,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:19,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:19,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:19,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:19,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:19,613 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:19,613 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:19,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:19,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:19,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:19,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-27 03:55:19,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2984a99a[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:19,614 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-27 03:55:19,614 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:19,615 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:19,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:19,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:19,615 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-27 03:55:19,615 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-27 03:55:19,615 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:19,616 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:19,616 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:19,618 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:19,618 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-27 03:55:19,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:19,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:19,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:19,618 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:19,618 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-27 03:55:19,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:19,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:19,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:19,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:19,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:19,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:19,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:19,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:19,620 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-27 03:55:19,620 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-27 03:55:19,620 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:19,620 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:19,620 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-27 03:55:19,620 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:19,622 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:19,622 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:19,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:19,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:19,622 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:19,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:19,622 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:19,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:19,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:19,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:19,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:19,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:19,623 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:19,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:19,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:19,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:19,624 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:19,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:19,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:19,626 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:19,626 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:19,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:19,626 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:19,626 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:19,626 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:19,626 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:19,626 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-27 03:55:19,626 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:19,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:19,626 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:19,626 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-27 03:55:19,626 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:19,627 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:19,626 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:19,627 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:19,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:19,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:19,627 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:19,727 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:19,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:19,729 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:19,729 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:19,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:19,732 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:19,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-27 03:55:19,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-27 03:55:19,734 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-27 03:55:19,734 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:19,734 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-27 03:55:19,734 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-27 03:55:19,735 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:19,735 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:19,736 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:19,736 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:19,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:19,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:19,736 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:19,736 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:19,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:19,736 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:19,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:19,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:19,737 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-27 03:55:19,737 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-27 03:55:19,737 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-27 03:55:19,737 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-27 03:55:19,737 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:19,737 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-27 03:55:19,737 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:19,738 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. started... 2023-03-27 03:55:19,738 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:19,738 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-27 03:55:19,738 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. started... 2023-03-27 03:55:19,738 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:19,738 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 9ce98af924f76608e57db6d1e9c69a22 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-27 03:55:19,738 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7bab4d71becda434aedcb5da6cc18929 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-27 03:55:19,738 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:19,738 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. started... 2023-03-27 03:55:19,738 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 72b74f7fa0bbf3322278037ab0d1d7f8 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-27 03:55:20,181 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032766efe68eade64437a36310927d82f292 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023032766efe68eade64437a36310927d82f292 2023-03-27 03:55:20,187 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032793f94520e8e3464297f296e1b526a62a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023032793f94520e8e3464297f296e1b526a62a 2023-03-27 03:55:20,192 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032780c8015b7a6a49fc8a811c5d7fe763d8 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023032780c8015b7a6a49fc8a811c5d7fe763d8 2023-03-27 03:55:20,587 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family1/8f483542fae3435388279c7c1249764b 2023-03-27 03:55:20,593 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family1/a7e3051bfba44fa2821afa6c2dd39032 2023-03-27 03:55:20,603 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family1/c87c2aa9fcc340bc8558ce66f105c0d0 2023-03-27 03:55:20,616 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230327b3978e53dd834af8a9053c79c3556a20 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230327b3978e53dd834af8a9053c79c3556a20 2023-03-27 03:55:21,010 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032714e7e83c42e34b4aa1d97aee55fba691 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023032714e7e83c42e34b4aa1d97aee55fba691 2023-03-27 03:55:21,023 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032787b8511f90ab442080c3065a9d5b2408 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023032787b8511f90ab442080c3065a9d5b2408 2023-03-27 03:55:21,030 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family2/dc47b7d13f264c1f93acb1986c318032 2023-03-27 03:55:21,034 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family1/a7e3051bfba44fa2821afa6c2dd39032 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/a7e3051bfba44fa2821afa6c2dd39032 2023-03-27 03:55:21,039 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family2/76911a9f9a754d2591e55e70d549ae8c 2023-03-27 03:55:21,040 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/a7e3051bfba44fa2821afa6c2dd39032, entries=4, sequenceid=9, filesize=5.4 K 2023-03-27 03:55:21,041 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family2/dc47b7d13f264c1f93acb1986c318032 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/dc47b7d13f264c1f93acb1986c318032 2023-03-27 03:55:21,046 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family1/c87c2aa9fcc340bc8558ce66f105c0d0 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/c87c2aa9fcc340bc8558ce66f105c0d0 2023-03-27 03:55:21,051 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/c87c2aa9fcc340bc8558ce66f105c0d0, entries=4, sequenceid=9, filesize=5.4 K 2023-03-27 03:55:21,052 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family2/76911a9f9a754d2591e55e70d549ae8c as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/76911a9f9a754d2591e55e70d549ae8c 2023-03-27 03:55:21,055 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/dc47b7d13f264c1f93acb1986c318032, entries=2, sequenceid=9, filesize=5.1 K 2023-03-27 03:55:21,056 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 72b74f7fa0bbf3322278037ab0d1d7f8 in 1318ms, sequenceid=9, compaction requested=false 2023-03-27 03:55:21,056 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 72b74f7fa0bbf3322278037ab0d1d7f8: 2023-03-27 03:55:21,056 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:21,056 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-27 03:55:21,056 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/76911a9f9a754d2591e55e70d549ae8c, entries=2, sequenceid=9, filesize=5.1 K 2023-03-27 03:55:21,057 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 7bab4d71becda434aedcb5da6cc18929 in 1319ms, sequenceid=9, compaction requested=false 2023-03-27 03:55:21,057 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7bab4d71becda434aedcb5da6cc18929: 2023-03-27 03:55:21,057 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:21,057 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-27 03:55:21,416 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family2/c9ffdd7294d84818aa9417670f805984 2023-03-27 03:55:21,421 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family1/8f483542fae3435388279c7c1249764b as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/8f483542fae3435388279c7c1249764b 2023-03-27 03:55:21,424 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/8f483542fae3435388279c7c1249764b, entries=4, sequenceid=9, filesize=5.4 K 2023-03-27 03:55:21,425 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family2/c9ffdd7294d84818aa9417670f805984 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/c9ffdd7294d84818aa9417670f805984 2023-03-27 03:55:21,429 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/c9ffdd7294d84818aa9417670f805984, entries=2, sequenceid=9, filesize=5.1 K 2023-03-27 03:55:21,430 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 9ce98af924f76608e57db6d1e9c69a22 in 1692ms, sequenceid=9, compaction requested=false 2023-03-27 03:55:21,430 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 9ce98af924f76608e57db6d1e9c69a22: 2023-03-27 03:55:21,430 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:21,431 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-27 03:55:21,431 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-27 03:55:21,431 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:21,431 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-27 03:55:21,431 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-27 03:55:21,433 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:21,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:21,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:21,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:21,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:21,434 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:21,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:21,434 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:21,434 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:21,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:21,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:21,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:21,435 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-27 03:55:21,435 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7acdd3db[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:21,435 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-27 03:55:21,435 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:21,437 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:21,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:21,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:21,437 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-27 03:55:21,437 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-27 03:55:21,437 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:21,437 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:21,437 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:21,442 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-27 03:55:21,442 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:21,442 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:21,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:21,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:21,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:21,442 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-27 03:55:21,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:21,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:21,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:21,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:21,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:21,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:21,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:21,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:21,444 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-27 03:55:21,444 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-27 03:55:21,444 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:21,444 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:21,444 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-27 03:55:21,444 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:21,447 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:21,447 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:21,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:21,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:21,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:21,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:21,447 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:21,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:21,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:21,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:21,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:21,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:21,448 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:21,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:21,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:21,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:21,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:21,449 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:21,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:21,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:21,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:21,454 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:21,454 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:21,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:21,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:21,454 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:21,454 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:21,454 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:21,454 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:21,454 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:21,454 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:21,454 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:21,454 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-27 03:55:21,454 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:21,454 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:21,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:21,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:21,455 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:21,455 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:21,555 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:21,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:21,557 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:21,557 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:21,558 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:21,561 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:21,564 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-27 03:55:21,564 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-27 03:55:21,564 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-27 03:55:21,564 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:21,564 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-27 03:55:21,564 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-27 03:55:21,565 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:21,565 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:21,566 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:21,566 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:21,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:21,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:21,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:21,566 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:21,566 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:21,567 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:21,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:21,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:21,567 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-27 03:55:21,567 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-27 03:55:21,567 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-27 03:55:21,567 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-27 03:55:21,567 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:21,567 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-27 03:55:21,568 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:21,568 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. started... 2023-03-27 03:55:21,568 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7bab4d71becda434aedcb5da6cc18929 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-27 03:55:21,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:21,576 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. started... 2023-03-27 03:55:21,576 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 9ce98af924f76608e57db6d1e9c69a22 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-27 03:55:21,580 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-27 03:55:21,580 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:21,580 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:21,580 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. started... 2023-03-27 03:55:21,580 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 72b74f7fa0bbf3322278037ab0d1d7f8 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-27 03:55:21,598 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032715ed5302d8bb49cfb21431c08e56c9cd to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023032715ed5302d8bb49cfb21431c08e56c9cd 2023-03-27 03:55:21,611 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230327c9e8daffb73c4dfda116da94c888eb56 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230327c9e8daffb73c4dfda116da94c888eb56 2023-03-27 03:55:21,997 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230327ae829be43cfa41028ccf3670377df3db to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230327ae829be43cfa41028ccf3670377df3db 2023-03-27 03:55:22,005 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family1/5482a6e4ced4444eb3630ce1ab3001f9 2023-03-27 03:55:22,018 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family1/e17ea5747d704ea0921a0e60c9af5a64 2023-03-27 03:55:22,021 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family1/a5a66da39cd3434f8447d270b2d94754 2023-03-27 03:55:22,093 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303277592a761bb5343eea9781f2ff5b574ce to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202303277592a761bb5343eea9781f2ff5b574ce 2023-03-27 03:55:22,104 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family2/a186431d23674d8bb1fa240f4efc9208 2023-03-27 03:55:22,110 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family1/a5a66da39cd3434f8447d270b2d94754 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/a5a66da39cd3434f8447d270b2d94754 2023-03-27 03:55:22,114 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/a5a66da39cd3434f8447d270b2d94754, entries=4, sequenceid=13, filesize=5.4 K 2023-03-27 03:55:22,115 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family2/a186431d23674d8bb1fa240f4efc9208 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/a186431d23674d8bb1fa240f4efc9208 2023-03-27 03:55:22,119 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/a186431d23674d8bb1fa240f4efc9208, entries=2, sequenceid=13, filesize=5.1 K 2023-03-27 03:55:22,120 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 9ce98af924f76608e57db6d1e9c69a22 in 544ms, sequenceid=13, compaction requested=true 2023-03-27 03:55:22,120 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 9ce98af924f76608e57db6d1e9c69a22: 2023-03-27 03:55:22,120 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:22,120 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-27 03:55:22,481 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230327268cca0a70cb4cca99ddb1884b486989 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230327268cca0a70cb4cca99ddb1884b486989 2023-03-27 03:55:22,489 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family2/1dabacb8b3b149eeb2eb23740a4a0195 2023-03-27 03:55:22,493 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family1/5482a6e4ced4444eb3630ce1ab3001f9 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/5482a6e4ced4444eb3630ce1ab3001f9 2023-03-27 03:55:22,493 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303278b835473a77b4215b0da0a866cb0193d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202303278b835473a77b4215b0da0a866cb0193d 2023-03-27 03:55:22,497 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/5482a6e4ced4444eb3630ce1ab3001f9, entries=4, sequenceid=13, filesize=5.4 K 2023-03-27 03:55:22,498 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family2/43aa64ed29374656bbd548fe3dbccf48 2023-03-27 03:55:22,499 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family2/1dabacb8b3b149eeb2eb23740a4a0195 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/1dabacb8b3b149eeb2eb23740a4a0195 2023-03-27 03:55:22,502 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family1/e17ea5747d704ea0921a0e60c9af5a64 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/e17ea5747d704ea0921a0e60c9af5a64 2023-03-27 03:55:22,503 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/1dabacb8b3b149eeb2eb23740a4a0195, entries=2, sequenceid=13, filesize=5.1 K 2023-03-27 03:55:22,504 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 7bab4d71becda434aedcb5da6cc18929 in 936ms, sequenceid=13, compaction requested=true 2023-03-27 03:55:22,504 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7bab4d71becda434aedcb5da6cc18929: 2023-03-27 03:55:22,504 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:22,504 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-27 03:55:22,506 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/e17ea5747d704ea0921a0e60c9af5a64, entries=4, sequenceid=13, filesize=5.4 K 2023-03-27 03:55:22,506 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family2/43aa64ed29374656bbd548fe3dbccf48 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/43aa64ed29374656bbd548fe3dbccf48 2023-03-27 03:55:22,510 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/43aa64ed29374656bbd548fe3dbccf48, entries=2, sequenceid=13, filesize=5.1 K 2023-03-27 03:55:22,511 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 72b74f7fa0bbf3322278037ab0d1d7f8 in 931ms, sequenceid=13, compaction requested=true 2023-03-27 03:55:22,511 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 72b74f7fa0bbf3322278037ab0d1d7f8: 2023-03-27 03:55:22,511 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:22,511 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-27 03:55:22,511 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-27 03:55:22,511 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:22,511 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-27 03:55:22,511 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-27 03:55:22,513 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:22,513 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:22,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:22,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:22,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:22,513 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:22,513 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:22,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:22,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:22,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:22,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:22,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:22,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-27 03:55:22,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6cd490c3[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:22,515 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-27 03:55:22,515 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:22,519 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:22,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:22,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:22,519 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-27 03:55:22,519 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-27 03:55:22,519 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:22,519 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:22,519 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:22,520 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:22,520 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-27 03:55:22,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:22,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:22,520 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:22,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:22,521 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-27 03:55:22,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:22,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:22,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:22,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:22,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:22,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:22,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:22,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:22,523 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-27 03:55:22,523 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-27 03:55:22,523 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:22,523 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:22,523 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-27 03:55:22,523 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:22,525 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:22,525 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:22,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:22,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:22,525 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:22,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:22,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:22,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:22,525 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:22,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:22,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:22,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:22,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:22,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:22,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:22,526 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:22,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:22,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:22,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:22,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:22,528 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:22,528 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:22,528 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:22,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:22,528 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:22,528 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:22,528 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:22,528 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:22,528 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:22,528 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:22,528 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:22,528 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-27 03:55:22,529 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:22,529 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-27 03:55:22,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:22,529 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:22,529 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:22,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:22,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:22,629 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:22,630 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:22,631 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:22,631 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:22,632 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:22,634 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:22,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-27 03:55:22,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-27 03:55:22,636 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-27 03:55:22,636 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:22,636 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-27 03:55:22,636 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-27 03:55:22,636 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:22,636 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:22,638 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:22,638 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:22,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:22,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:22,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:22,639 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:22,639 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:22,639 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:22,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:22,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:22,640 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-27 03:55:22,640 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-27 03:55:22,640 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-27 03:55:22,640 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-27 03:55:22,640 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:22,640 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-27 03:55:22,640 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:22,640 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. started... 2023-03-27 03:55:22,640 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-27 03:55:22,640 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:22,640 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:22,641 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7bab4d71becda434aedcb5da6cc18929 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-27 03:55:22,641 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:22,641 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. started... 2023-03-27 03:55:22,641 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. started... 2023-03-27 03:55:22,641 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 9ce98af924f76608e57db6d1e9c69a22 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-27 03:55:22,641 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 72b74f7fa0bbf3322278037ab0d1d7f8 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-27 03:55:22,665 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032788f62ab4a7ee418fbb7417e13c6de0dd to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023032788f62ab4a7ee418fbb7417e13c6de0dd 2023-03-27 03:55:22,675 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family1/c0a25c7b55384668a0300ef1f1285c6f 2023-03-27 03:55:23,062 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032713fc9d7a1eaf42209bbfdda0f76eda13 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023032713fc9d7a1eaf42209bbfdda0f76eda13 2023-03-27 03:55:23,062 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303278ff5077dd250480fbc0bb38c6f29173e to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202303278ff5077dd250480fbc0bb38c6f29173e 2023-03-27 03:55:23,069 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family1/c7cec720a2144d0eaa1f4d9b2663859f 2023-03-27 03:55:23,072 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family1/90f0de71963546f28563aaed5d2817a8 2023-03-27 03:55:23,092 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032752ed6aad89544e6d98d6b24346645ca1 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023032752ed6aad89544e6d98d6b24346645ca1 2023-03-27 03:55:23,097 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family2/eae7a54eb7504ca0935439e5fc7a9b29 2023-03-27 03:55:23,101 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family1/c0a25c7b55384668a0300ef1f1285c6f as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/c0a25c7b55384668a0300ef1f1285c6f 2023-03-27 03:55:23,105 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/c0a25c7b55384668a0300ef1f1285c6f, entries=4, sequenceid=17, filesize=5.4 K 2023-03-27 03:55:23,105 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family2/eae7a54eb7504ca0935439e5fc7a9b29 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/eae7a54eb7504ca0935439e5fc7a9b29 2023-03-27 03:55:23,109 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/eae7a54eb7504ca0935439e5fc7a9b29, entries=2, sequenceid=17, filesize=5.1 K 2023-03-27 03:55:23,110 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 72b74f7fa0bbf3322278037ab0d1d7f8 in 469ms, sequenceid=17, compaction requested=true 2023-03-27 03:55:23,110 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 72b74f7fa0bbf3322278037ab0d1d7f8: 2023-03-27 03:55:23,110 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:23,110 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-27 03:55:23,170 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-27 03:55:23,495 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032775493a1022b6405bab7ecc109d572030 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023032775493a1022b6405bab7ecc109d572030 2023-03-27 03:55:23,499 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303275ae5ef5425d849bd9ca4252e3a41c533 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202303275ae5ef5425d849bd9ca4252e3a41c533 2023-03-27 03:55:23,500 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family2/4024039626004fe1912de582800a28d2 2023-03-27 03:55:23,504 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family1/c7cec720a2144d0eaa1f4d9b2663859f as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/c7cec720a2144d0eaa1f4d9b2663859f 2023-03-27 03:55:23,505 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family2/3d067f5b9dbb4bde9a42abf61acbac64 2023-03-27 03:55:23,508 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/c7cec720a2144d0eaa1f4d9b2663859f, entries=4, sequenceid=17, filesize=5.4 K 2023-03-27 03:55:23,511 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family2/4024039626004fe1912de582800a28d2 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/4024039626004fe1912de582800a28d2 2023-03-27 03:55:23,511 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family1/90f0de71963546f28563aaed5d2817a8 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/90f0de71963546f28563aaed5d2817a8 2023-03-27 03:55:23,515 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/4024039626004fe1912de582800a28d2, entries=2, sequenceid=17, filesize=5.1 K 2023-03-27 03:55:23,515 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/90f0de71963546f28563aaed5d2817a8, entries=4, sequenceid=17, filesize=5.4 K 2023-03-27 03:55:23,516 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 7bab4d71becda434aedcb5da6cc18929 in 876ms, sequenceid=17, compaction requested=true 2023-03-27 03:55:23,516 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7bab4d71becda434aedcb5da6cc18929: 2023-03-27 03:55:23,516 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:23,516 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-27 03:55:23,516 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family2/3d067f5b9dbb4bde9a42abf61acbac64 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/3d067f5b9dbb4bde9a42abf61acbac64 2023-03-27 03:55:23,520 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/3d067f5b9dbb4bde9a42abf61acbac64, entries=2, sequenceid=17, filesize=5.1 K 2023-03-27 03:55:23,521 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 9ce98af924f76608e57db6d1e9c69a22 in 880ms, sequenceid=17, compaction requested=true 2023-03-27 03:55:23,521 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 9ce98af924f76608e57db6d1e9c69a22: 2023-03-27 03:55:23,521 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:23,521 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-27 03:55:23,521 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-27 03:55:23,521 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:23,521 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-27 03:55:23,521 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-27 03:55:23,522 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:23,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:23,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:23,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:23,522 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:23,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:23,523 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:23,523 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:23,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:23,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:23,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:23,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:23,524 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-27 03:55:23,524 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b463046[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:23,524 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-27 03:55:23,524 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:23,526 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:23,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:23,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:23,526 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-27 03:55:23,526 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-27 03:55:23,526 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:23,526 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:23,526 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:23,527 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:23,527 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-27 03:55:23,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:23,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:23,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:23,527 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:23,527 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-27 03:55:23,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:23,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:23,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:23,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:23,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:23,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:23,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:23,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:23,530 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-27 03:55:23,530 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-27 03:55:23,530 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:23,530 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:23,530 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-27 03:55:23,530 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:23,532 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:23,532 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:23,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:23,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:23,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:23,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:23,533 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:23,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:23,533 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:23,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:23,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:23,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:23,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:23,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:23,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:23,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:23,535 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:23,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:23,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:23,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:23,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:23,537 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:23,537 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:23,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:23,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-27 03:55:23,538 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:23,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:23,537 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:23,537 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:23,538 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-03-27 03:55:23,538 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:23,538 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:23,538 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:23,538 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:23,538 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:23,538 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:23,538 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:23,538 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:23,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:23,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:23,638 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:23,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:23,651 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-27 03:55:23,663 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-27 03:55:23,680 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-27 03:55:23,683 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-27 03:55:23,717 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:23,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-03-27 03:55:23,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-03-27 03:55:23,719 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-03-27 03:55:23,719 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:23,719 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-03-27 03:55:23,719 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-03-27 03:55:23,720 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:23,720 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:23,721 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:23,721 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:23,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:23,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:23,721 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:23,721 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:23,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:23,722 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:23,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:23,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:23,722 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-03-27 03:55:23,722 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-03-27 03:55:23,722 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-03-27 03:55:23,722 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-03-27 03:55:23,722 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:23,723 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-03-27 03:55:23,723 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:23,723 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. started... 2023-03-27 03:55:23,723 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-27 03:55:23,723 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:23,723 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:23,723 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 7bab4d71becda434aedcb5da6cc18929 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-27 03:55:23,723 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. started... 2023-03-27 03:55:23,723 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:23,723 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 9ce98af924f76608e57db6d1e9c69a22 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-27 03:55:23,723 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. started... 2023-03-27 03:55:23,723 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 72b74f7fa0bbf3322278037ab0d1d7f8 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-27 03:55:23,764 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family1/717e960726934c80aa4afe5f6529c232 2023-03-27 03:55:23,764 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family1/74bbd65530fa46288b0cedc9ae6706a3 2023-03-27 03:55:23,766 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family1/93f8becd91e64af79d185ba42c36245e 2023-03-27 03:55:23,768 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 717e960726934c80aa4afe5f6529c232 2023-03-27 03:55:23,770 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 74bbd65530fa46288b0cedc9ae6706a3 2023-03-27 03:55:23,770 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 93f8becd91e64af79d185ba42c36245e 2023-03-27 03:55:23,804 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family2/03c02340cda247dbbf39dbc5747dabac 2023-03-27 03:55:23,808 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03c02340cda247dbbf39dbc5747dabac 2023-03-27 03:55:23,808 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family1/93f8becd91e64af79d185ba42c36245e as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/93f8becd91e64af79d185ba42c36245e 2023-03-27 03:55:23,812 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 93f8becd91e64af79d185ba42c36245e 2023-03-27 03:55:23,812 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/93f8becd91e64af79d185ba42c36245e, entries=3, sequenceid=23, filesize=5.0 K 2023-03-27 03:55:23,813 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family2/03c02340cda247dbbf39dbc5747dabac as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/03c02340cda247dbbf39dbc5747dabac 2023-03-27 03:55:23,816 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03c02340cda247dbbf39dbc5747dabac 2023-03-27 03:55:23,816 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/03c02340cda247dbbf39dbc5747dabac, entries=1, sequenceid=23, filesize=5.0 K 2023-03-27 03:55:23,817 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 7bab4d71becda434aedcb5da6cc18929 in 94ms, sequenceid=23, compaction requested=true 2023-03-27 03:55:23,817 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 7bab4d71becda434aedcb5da6cc18929: 2023-03-27 03:55:23,817 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:23,817 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-27 03:55:24,197 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family2/ab76f6080ca5446881a88d5804165b65 2023-03-27 03:55:24,201 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ab76f6080ca5446881a88d5804165b65 2023-03-27 03:55:24,202 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family1/717e960726934c80aa4afe5f6529c232 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/717e960726934c80aa4afe5f6529c232 2023-03-27 03:55:24,205 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 717e960726934c80aa4afe5f6529c232 2023-03-27 03:55:24,206 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/717e960726934c80aa4afe5f6529c232, entries=3, sequenceid=23, filesize=5.0 K 2023-03-27 03:55:24,206 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family2/ab76f6080ca5446881a88d5804165b65 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/ab76f6080ca5446881a88d5804165b65 2023-03-27 03:55:24,210 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ab76f6080ca5446881a88d5804165b65 2023-03-27 03:55:24,210 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/ab76f6080ca5446881a88d5804165b65, entries=1, sequenceid=23, filesize=5.0 K 2023-03-27 03:55:24,211 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 72b74f7fa0bbf3322278037ab0d1d7f8 in 488ms, sequenceid=23, compaction requested=true 2023-03-27 03:55:24,211 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 72b74f7fa0bbf3322278037ab0d1d7f8: 2023-03-27 03:55:24,211 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:24,211 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-27 03:55:24,597 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family2/c6b00f1eb0b74be99bf97f31d8a90677 2023-03-27 03:55:24,601 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c6b00f1eb0b74be99bf97f31d8a90677 2023-03-27 03:55:24,602 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family1/74bbd65530fa46288b0cedc9ae6706a3 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/74bbd65530fa46288b0cedc9ae6706a3 2023-03-27 03:55:24,613 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 74bbd65530fa46288b0cedc9ae6706a3 2023-03-27 03:55:24,613 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/74bbd65530fa46288b0cedc9ae6706a3, entries=3, sequenceid=23, filesize=5.0 K 2023-03-27 03:55:24,614 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family2/c6b00f1eb0b74be99bf97f31d8a90677 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/c6b00f1eb0b74be99bf97f31d8a90677 2023-03-27 03:55:24,618 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c6b00f1eb0b74be99bf97f31d8a90677 2023-03-27 03:55:24,618 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/c6b00f1eb0b74be99bf97f31d8a90677, entries=1, sequenceid=23, filesize=5.0 K 2023-03-27 03:55:24,620 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 9ce98af924f76608e57db6d1e9c69a22 in 897ms, sequenceid=23, compaction requested=true 2023-03-27 03:55:24,620 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 9ce98af924f76608e57db6d1e9c69a22: 2023-03-27 03:55:24,620 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:24,620 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-27 03:55:24,620 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-27 03:55:24,620 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:24,620 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-03-27 03:55:24,620 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-03-27 03:55:24,622 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:24,622 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:24,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:24,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:24,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:24,622 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:24,622 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:24,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:24,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:24,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:24,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:24,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:24,624 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-03-27 03:55:24,624 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@470288cd[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:24,624 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-03-27 03:55:24,624 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:24,626 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:24,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:24,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:24,626 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-03-27 03:55:24,626 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-03-27 03:55:24,626 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:24,626 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:24,627 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:24,630 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-03-27 03:55:24,630 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:24,630 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:24,630 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-03-27 03:55:24,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:24,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:24,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:24,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:24,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:24,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:24,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:24,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:24,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:24,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:24,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:24,633 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-03-27 03:55:24,633 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-03-27 03:55:24,633 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:24,633 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:24,634 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-03-27 03:55:24,634 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:24,635 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:24,635 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:24,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:24,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:24,635 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:24,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:24,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:24,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:24,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:24,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:24,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:24,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:24,636 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:24,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:24,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:24,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:24,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:24,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:24,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-03-27 03:55:24,638 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:24,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:24,641 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:24,641 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:24,641 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:24,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:24,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:24,641 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:24,641 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:24,641 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-03-27 03:55:24,641 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-03-27 03:55:24,641 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:24,641 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:24,642 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:24,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:24,642 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-03-27 03:55:24,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:24,642 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-03-27 03:55:24,642 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:24,642 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:24,742 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:24,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:24,743 DEBUG [Listener at localhost/36131] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-27 03:55:24,743 DEBUG [Listener at localhost/36131] regionserver.HStore(1912): 7bab4d71becda434aedcb5da6cc18929/family1 is initiating major compaction (all files) 2023-03-27 03:55:24,744 INFO [Listener at localhost/36131] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-27 03:55:24,744 INFO [Listener at localhost/36131] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-27 03:55:24,744 INFO [Listener at localhost/36131] regionserver.HRegion(2259): Starting compaction of 7bab4d71becda434aedcb5da6cc18929/family1 in ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:24,744 INFO [Listener at localhost/36131] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/0d8e89097f6e4d68a18a2dbd04ed6dfb, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/c87c2aa9fcc340bc8558ce66f105c0d0, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/5482a6e4ced4444eb3630ce1ab3001f9, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/c7cec720a2144d0eaa1f4d9b2663859f, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/93f8becd91e64af79d185ba42c36245e] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp, totalSize=26.6 K 2023-03-27 03:55:24,745 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 0d8e89097f6e4d68a18a2dbd04ed6dfb, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889318624 2023-03-27 03:55:24,745 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting c87c2aa9fcc340bc8558ce66f105c0d0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889319729 2023-03-27 03:55:24,746 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 5482a6e4ced4444eb3630ce1ab3001f9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889321557 2023-03-27 03:55:24,746 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting c7cec720a2144d0eaa1f4d9b2663859f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889322631 2023-03-27 03:55:24,746 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 93f8becd91e64af79d185ba42c36245e, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-27 03:55:24,764 INFO [Listener at localhost/36131] throttle.PressureAwareThroughputController(145): 7bab4d71becda434aedcb5da6cc18929#family1#compaction#150 average throughput is 0.41 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:24,773 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023032764d9cf497a8d4896a788b36c51b373ce_del 2023-03-27 03:55:24,773 INFO [Listener at localhost/36131] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032764d9cf497a8d4896a788b36c51b373ce_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023032764d9cf497a8d4896a788b36c51b373ce_del 2023-03-27 03:55:25,187 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 51db1e2652dc40acb2060c357aeb4b4d 2023-03-27 03:55:25,187 DEBUG [Listener at localhost/36131] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family1/51db1e2652dc40acb2060c357aeb4b4d as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/51db1e2652dc40acb2060c357aeb4b4d 2023-03-27 03:55:25,191 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 51db1e2652dc40acb2060c357aeb4b4d 2023-03-27 03:55:25,192 INFO [Listener at localhost/36131] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 7bab4d71becda434aedcb5da6cc18929/family1 of 7bab4d71becda434aedcb5da6cc18929 into 51db1e2652dc40acb2060c357aeb4b4d(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:55:25,192 DEBUG [Listener at localhost/36131] regionserver.HRegion(2289): Compaction status journal for 7bab4d71becda434aedcb5da6cc18929: 2023-03-27 03:55:25,192 DEBUG [Listener at localhost/36131] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-27 03:55:25,193 DEBUG [Listener at localhost/36131] regionserver.HStore(1912): 7bab4d71becda434aedcb5da6cc18929/family2 is initiating major compaction (all files) 2023-03-27 03:55:25,193 INFO [Listener at localhost/36131] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-27 03:55:25,193 INFO [Listener at localhost/36131] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-27 03:55:25,193 INFO [Listener at localhost/36131] regionserver.HRegion(2259): Starting compaction of 7bab4d71becda434aedcb5da6cc18929/family2 in ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:25,193 INFO [Listener at localhost/36131] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/4a249aca31574b1f8cc171212769055b, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/76911a9f9a754d2591e55e70d549ae8c, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/1dabacb8b3b149eeb2eb23740a4a0195, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/4024039626004fe1912de582800a28d2, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/03c02340cda247dbbf39dbc5747dabac] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp, totalSize=25.3 K 2023-03-27 03:55:25,193 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 4a249aca31574b1f8cc171212769055b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889318624 2023-03-27 03:55:25,194 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 76911a9f9a754d2591e55e70d549ae8c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889319729 2023-03-27 03:55:25,194 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 1dabacb8b3b149eeb2eb23740a4a0195, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889321557 2023-03-27 03:55:25,195 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 4024039626004fe1912de582800a28d2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889322631 2023-03-27 03:55:25,195 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 03c02340cda247dbbf39dbc5747dabac, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-27 03:55:25,199 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/0d8e89097f6e4d68a18a2dbd04ed6dfb, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/c87c2aa9fcc340bc8558ce66f105c0d0, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/5482a6e4ced4444eb3630ce1ab3001f9, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/c7cec720a2144d0eaa1f4d9b2663859f, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/93f8becd91e64af79d185ba42c36245e] to archive 2023-03-27 03:55:25,199 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:25,202 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/0d8e89097f6e4d68a18a2dbd04ed6dfb to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/0d8e89097f6e4d68a18a2dbd04ed6dfb 2023-03-27 03:55:25,204 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/c87c2aa9fcc340bc8558ce66f105c0d0 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/c87c2aa9fcc340bc8558ce66f105c0d0 2023-03-27 03:55:25,205 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/5482a6e4ced4444eb3630ce1ab3001f9 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/5482a6e4ced4444eb3630ce1ab3001f9 2023-03-27 03:55:25,207 INFO [Listener at localhost/36131] throttle.PressureAwareThroughputController(145): 7bab4d71becda434aedcb5da6cc18929#family2#compaction#151 average throughput is 0.25 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:25,207 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/c7cec720a2144d0eaa1f4d9b2663859f to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/c7cec720a2144d0eaa1f4d9b2663859f 2023-03-27 03:55:25,209 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/93f8becd91e64af79d185ba42c36245e to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/93f8becd91e64af79d185ba42c36245e 2023-03-27 03:55:25,615 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230327c4bf5336cf87472394f6b3689ce10f6c_del 2023-03-27 03:55:25,616 INFO [Listener at localhost/36131] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230327c4bf5336cf87472394f6b3689ce10f6c_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230327c4bf5336cf87472394f6b3689ce10f6c_del 2023-03-27 03:55:26,033 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0c3c5ce86ba4e68a9be8e470f104665 2023-03-27 03:55:26,034 DEBUG [Listener at localhost/36131] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/.tmp/family2/f0c3c5ce86ba4e68a9be8e470f104665 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/f0c3c5ce86ba4e68a9be8e470f104665 2023-03-27 03:55:26,038 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0c3c5ce86ba4e68a9be8e470f104665 2023-03-27 03:55:26,039 INFO [Listener at localhost/36131] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 7bab4d71becda434aedcb5da6cc18929/family2 of 7bab4d71becda434aedcb5da6cc18929 into f0c3c5ce86ba4e68a9be8e470f104665(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:55:26,039 DEBUG [Listener at localhost/36131] regionserver.HRegion(2289): Compaction status journal for 7bab4d71becda434aedcb5da6cc18929: 2023-03-27 03:55:26,039 DEBUG [Listener at localhost/36131] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-27 03:55:26,039 DEBUG [Listener at localhost/36131] regionserver.HStore(1912): 9ce98af924f76608e57db6d1e9c69a22/family1 is initiating major compaction (all files) 2023-03-27 03:55:26,040 INFO [Listener at localhost/36131] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-27 03:55:26,040 INFO [Listener at localhost/36131] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-27 03:55:26,041 INFO [Listener at localhost/36131] regionserver.HRegion(2259): Starting compaction of 9ce98af924f76608e57db6d1e9c69a22/family1 in ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:26,041 INFO [Listener at localhost/36131] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/63d80d62e7cc4f1598413736a7da0d92, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/8f483542fae3435388279c7c1249764b, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/a5a66da39cd3434f8447d270b2d94754, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/90f0de71963546f28563aaed5d2817a8, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/74bbd65530fa46288b0cedc9ae6706a3] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp, totalSize=26.6 K 2023-03-27 03:55:26,042 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 63d80d62e7cc4f1598413736a7da0d92, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889318625 2023-03-27 03:55:26,042 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 8f483542fae3435388279c7c1249764b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889319729 2023-03-27 03:55:26,042 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting a5a66da39cd3434f8447d270b2d94754, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889321557 2023-03-27 03:55:26,043 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 90f0de71963546f28563aaed5d2817a8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889322631 2023-03-27 03:55:26,043 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 74bbd65530fa46288b0cedc9ae6706a3, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-27 03:55:26,051 INFO [Listener at localhost/36131] throttle.PressureAwareThroughputController(145): 9ce98af924f76608e57db6d1e9c69a22#family1#compaction#152 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:26,091 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/4a249aca31574b1f8cc171212769055b, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/76911a9f9a754d2591e55e70d549ae8c, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/1dabacb8b3b149eeb2eb23740a4a0195, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/4024039626004fe1912de582800a28d2, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/03c02340cda247dbbf39dbc5747dabac] to archive 2023-03-27 03:55:26,092 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:26,094 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/4a249aca31574b1f8cc171212769055b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/4a249aca31574b1f8cc171212769055b 2023-03-27 03:55:26,095 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/76911a9f9a754d2591e55e70d549ae8c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/76911a9f9a754d2591e55e70d549ae8c 2023-03-27 03:55:26,096 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/1dabacb8b3b149eeb2eb23740a4a0195 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/1dabacb8b3b149eeb2eb23740a4a0195 2023-03-27 03:55:26,097 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/4024039626004fe1912de582800a28d2 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/4024039626004fe1912de582800a28d2 2023-03-27 03:55:26,098 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/03c02340cda247dbbf39dbc5747dabac to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family2/03c02340cda247dbbf39dbc5747dabac 2023-03-27 03:55:26,466 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230327c76554099b7f4e9aa64f61a55973f42b_del 2023-03-27 03:55:26,466 INFO [Listener at localhost/36131] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230327c76554099b7f4e9aa64f61a55973f42b_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230327c76554099b7f4e9aa64f61a55973f42b_del 2023-03-27 03:55:26,881 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7f94000d81584633810b3009a5570183 2023-03-27 03:55:26,882 DEBUG [Listener at localhost/36131] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family1/7f94000d81584633810b3009a5570183 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/7f94000d81584633810b3009a5570183 2023-03-27 03:55:26,888 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7f94000d81584633810b3009a5570183 2023-03-27 03:55:26,889 INFO [Listener at localhost/36131] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 9ce98af924f76608e57db6d1e9c69a22/family1 of 9ce98af924f76608e57db6d1e9c69a22 into 7f94000d81584633810b3009a5570183(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:55:26,889 DEBUG [Listener at localhost/36131] regionserver.HRegion(2289): Compaction status journal for 9ce98af924f76608e57db6d1e9c69a22: 2023-03-27 03:55:26,890 DEBUG [Listener at localhost/36131] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-27 03:55:26,890 DEBUG [Listener at localhost/36131] regionserver.HStore(1912): 9ce98af924f76608e57db6d1e9c69a22/family2 is initiating major compaction (all files) 2023-03-27 03:55:26,890 INFO [Listener at localhost/36131] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-27 03:55:26,890 INFO [Listener at localhost/36131] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-27 03:55:26,890 INFO [Listener at localhost/36131] regionserver.HRegion(2259): Starting compaction of 9ce98af924f76608e57db6d1e9c69a22/family2 in ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:26,890 INFO [Listener at localhost/36131] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/7f4b087b465f43aa8632446e7c860dbe, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/c9ffdd7294d84818aa9417670f805984, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/a186431d23674d8bb1fa240f4efc9208, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/3d067f5b9dbb4bde9a42abf61acbac64, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/c6b00f1eb0b74be99bf97f31d8a90677] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp, totalSize=25.3 K 2023-03-27 03:55:26,891 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 7f4b087b465f43aa8632446e7c860dbe, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889318625 2023-03-27 03:55:26,891 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting c9ffdd7294d84818aa9417670f805984, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889319729 2023-03-27 03:55:26,892 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting a186431d23674d8bb1fa240f4efc9208, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889321557 2023-03-27 03:55:26,892 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 3d067f5b9dbb4bde9a42abf61acbac64, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889322631 2023-03-27 03:55:26,893 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting c6b00f1eb0b74be99bf97f31d8a90677, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-27 03:55:26,898 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/63d80d62e7cc4f1598413736a7da0d92, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/8f483542fae3435388279c7c1249764b, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/a5a66da39cd3434f8447d270b2d94754, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/90f0de71963546f28563aaed5d2817a8, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/74bbd65530fa46288b0cedc9ae6706a3] to archive 2023-03-27 03:55:26,899 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:26,900 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/63d80d62e7cc4f1598413736a7da0d92 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/63d80d62e7cc4f1598413736a7da0d92 2023-03-27 03:55:26,902 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/8f483542fae3435388279c7c1249764b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/8f483542fae3435388279c7c1249764b 2023-03-27 03:55:26,903 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/a5a66da39cd3434f8447d270b2d94754 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/a5a66da39cd3434f8447d270b2d94754 2023-03-27 03:55:26,905 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/90f0de71963546f28563aaed5d2817a8 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/90f0de71963546f28563aaed5d2817a8 2023-03-27 03:55:26,906 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/74bbd65530fa46288b0cedc9ae6706a3 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/74bbd65530fa46288b0cedc9ae6706a3 2023-03-27 03:55:26,908 INFO [Listener at localhost/36131] throttle.PressureAwareThroughputController(145): 9ce98af924f76608e57db6d1e9c69a22#family2#compaction#153 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:27,329 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303277e6f3021baa54931a10715e9f8cc02b4_del 2023-03-27 03:55:27,330 INFO [Listener at localhost/36131] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303277e6f3021baa54931a10715e9f8cc02b4_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202303277e6f3021baa54931a10715e9f8cc02b4_del 2023-03-27 03:55:28,155 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5645633016704baf927342b9d27f6d6b 2023-03-27 03:55:28,156 DEBUG [Listener at localhost/36131] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/.tmp/family2/5645633016704baf927342b9d27f6d6b as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/5645633016704baf927342b9d27f6d6b 2023-03-27 03:55:28,160 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5645633016704baf927342b9d27f6d6b 2023-03-27 03:55:28,161 INFO [Listener at localhost/36131] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 9ce98af924f76608e57db6d1e9c69a22/family2 of 9ce98af924f76608e57db6d1e9c69a22 into 5645633016704baf927342b9d27f6d6b(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-27 03:55:28,161 DEBUG [Listener at localhost/36131] regionserver.HRegion(2289): Compaction status journal for 9ce98af924f76608e57db6d1e9c69a22: 2023-03-27 03:55:28,162 DEBUG [Listener at localhost/36131] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-27 03:55:28,162 DEBUG [Listener at localhost/36131] regionserver.HStore(1912): 72b74f7fa0bbf3322278037ab0d1d7f8/family1 is initiating major compaction (all files) 2023-03-27 03:55:28,162 INFO [Listener at localhost/36131] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-27 03:55:28,162 INFO [Listener at localhost/36131] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-27 03:55:28,163 INFO [Listener at localhost/36131] regionserver.HRegion(2259): Starting compaction of 72b74f7fa0bbf3322278037ab0d1d7f8/family1 in ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:28,164 INFO [Listener at localhost/36131] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/f29e92c205ff4ec4a1aa35defdb9c29c, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/a7e3051bfba44fa2821afa6c2dd39032, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/e17ea5747d704ea0921a0e60c9af5a64, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/c0a25c7b55384668a0300ef1f1285c6f, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/717e960726934c80aa4afe5f6529c232] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp, totalSize=26.6 K 2023-03-27 03:55:28,164 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting f29e92c205ff4ec4a1aa35defdb9c29c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889318626 2023-03-27 03:55:28,164 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting a7e3051bfba44fa2821afa6c2dd39032, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889319730 2023-03-27 03:55:28,165 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting e17ea5747d704ea0921a0e60c9af5a64, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889321558 2023-03-27 03:55:28,165 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting c0a25c7b55384668a0300ef1f1285c6f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889322632 2023-03-27 03:55:28,165 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 717e960726934c80aa4afe5f6529c232, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-27 03:55:28,175 INFO [Listener at localhost/36131] throttle.PressureAwareThroughputController(145): 72b74f7fa0bbf3322278037ab0d1d7f8#family1#compaction#154 average throughput is 0.55 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:28,182 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230327b692662b8ecb4a02bd084a945e7bcceb_del 2023-03-27 03:55:28,182 INFO [Listener at localhost/36131] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230327b692662b8ecb4a02bd084a945e7bcceb_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230327b692662b8ecb4a02bd084a945e7bcceb_del 2023-03-27 03:55:28,189 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/7f4b087b465f43aa8632446e7c860dbe, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/c9ffdd7294d84818aa9417670f805984, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/a186431d23674d8bb1fa240f4efc9208, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/3d067f5b9dbb4bde9a42abf61acbac64, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/c6b00f1eb0b74be99bf97f31d8a90677] to archive 2023-03-27 03:55:28,189 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:28,191 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/7f4b087b465f43aa8632446e7c860dbe to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/7f4b087b465f43aa8632446e7c860dbe 2023-03-27 03:55:28,192 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/c9ffdd7294d84818aa9417670f805984 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/c9ffdd7294d84818aa9417670f805984 2023-03-27 03:55:28,193 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/a186431d23674d8bb1fa240f4efc9208 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/a186431d23674d8bb1fa240f4efc9208 2023-03-27 03:55:28,194 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/3d067f5b9dbb4bde9a42abf61acbac64 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/3d067f5b9dbb4bde9a42abf61acbac64 2023-03-27 03:55:28,196 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/c6b00f1eb0b74be99bf97f31d8a90677 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family2/c6b00f1eb0b74be99bf97f31d8a90677 2023-03-27 03:55:28,598 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 36f824acb90a423d8bcdbdd52cb24529 2023-03-27 03:55:28,599 DEBUG [Listener at localhost/36131] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family1/36f824acb90a423d8bcdbdd52cb24529 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/36f824acb90a423d8bcdbdd52cb24529 2023-03-27 03:55:28,603 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 36f824acb90a423d8bcdbdd52cb24529 2023-03-27 03:55:28,604 INFO [Listener at localhost/36131] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 72b74f7fa0bbf3322278037ab0d1d7f8/family1 of 72b74f7fa0bbf3322278037ab0d1d7f8 into 36f824acb90a423d8bcdbdd52cb24529(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:55:28,604 DEBUG [Listener at localhost/36131] regionserver.HRegion(2289): Compaction status journal for 72b74f7fa0bbf3322278037ab0d1d7f8: 2023-03-27 03:55:28,604 DEBUG [Listener at localhost/36131] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-27 03:55:28,604 DEBUG [Listener at localhost/36131] regionserver.HStore(1912): 72b74f7fa0bbf3322278037ab0d1d7f8/family2 is initiating major compaction (all files) 2023-03-27 03:55:28,604 INFO [Listener at localhost/36131] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-27 03:55:28,604 INFO [Listener at localhost/36131] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-27 03:55:28,605 INFO [Listener at localhost/36131] regionserver.HRegion(2259): Starting compaction of 72b74f7fa0bbf3322278037ab0d1d7f8/family2 in ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:28,605 INFO [Listener at localhost/36131] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/2e74877b929c49099c6ed1b42da58dac, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/dc47b7d13f264c1f93acb1986c318032, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/43aa64ed29374656bbd548fe3dbccf48, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/eae7a54eb7504ca0935439e5fc7a9b29, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/ab76f6080ca5446881a88d5804165b65] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp, totalSize=25.3 K 2023-03-27 03:55:28,609 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 2e74877b929c49099c6ed1b42da58dac, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889318626 2023-03-27 03:55:28,610 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting dc47b7d13f264c1f93acb1986c318032, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889319730 2023-03-27 03:55:28,610 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 43aa64ed29374656bbd548fe3dbccf48, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889321558 2023-03-27 03:55:28,611 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting eae7a54eb7504ca0935439e5fc7a9b29, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889322632 2023-03-27 03:55:28,611 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting ab76f6080ca5446881a88d5804165b65, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-27 03:55:28,629 INFO [Listener at localhost/36131] throttle.PressureAwareThroughputController(145): 72b74f7fa0bbf3322278037ab0d1d7f8#family2#compaction#155 average throughput is 0.10 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:28,689 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/f29e92c205ff4ec4a1aa35defdb9c29c, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/a7e3051bfba44fa2821afa6c2dd39032, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/e17ea5747d704ea0921a0e60c9af5a64, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/c0a25c7b55384668a0300ef1f1285c6f, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/717e960726934c80aa4afe5f6529c232] to archive 2023-03-27 03:55:28,690 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:28,692 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/f29e92c205ff4ec4a1aa35defdb9c29c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/f29e92c205ff4ec4a1aa35defdb9c29c 2023-03-27 03:55:28,693 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/a7e3051bfba44fa2821afa6c2dd39032 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/a7e3051bfba44fa2821afa6c2dd39032 2023-03-27 03:55:28,694 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/e17ea5747d704ea0921a0e60c9af5a64 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/e17ea5747d704ea0921a0e60c9af5a64 2023-03-27 03:55:28,696 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/c0a25c7b55384668a0300ef1f1285c6f to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/c0a25c7b55384668a0300ef1f1285c6f 2023-03-27 03:55:28,697 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/717e960726934c80aa4afe5f6529c232 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/717e960726934c80aa4afe5f6529c232 2023-03-27 03:55:29,040 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202303276326bed0217947609de75922a5269523_del 2023-03-27 03:55:29,040 INFO [Listener at localhost/36131] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303276326bed0217947609de75922a5269523_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202303276326bed0217947609de75922a5269523_del 2023-03-27 03:55:29,057 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4feff3c330ce4815b4bcb93d4f88b087 2023-03-27 03:55:29,057 DEBUG [Listener at localhost/36131] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/.tmp/family2/4feff3c330ce4815b4bcb93d4f88b087 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/4feff3c330ce4815b4bcb93d4f88b087 2023-03-27 03:55:29,063 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4feff3c330ce4815b4bcb93d4f88b087 2023-03-27 03:55:29,064 INFO [Listener at localhost/36131] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 72b74f7fa0bbf3322278037ab0d1d7f8/family2 of 72b74f7fa0bbf3322278037ab0d1d7f8 into 4feff3c330ce4815b4bcb93d4f88b087(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:55:29,064 DEBUG [Listener at localhost/36131] regionserver.HRegion(2289): Compaction status journal for 72b74f7fa0bbf3322278037ab0d1d7f8: 2023-03-27 03:55:29,069 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-27 03:55:29,072 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-27 03:55:29,075 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-27 03:55:29,076 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-27 03:55:29,082 INFO [Listener at localhost/36131] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-27 03:55:29,095 INFO [Listener at localhost/36131] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-03-27 03:55:29,098 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/2e74877b929c49099c6ed1b42da58dac, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/dc47b7d13f264c1f93acb1986c318032, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/43aa64ed29374656bbd548fe3dbccf48, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/eae7a54eb7504ca0935439e5fc7a9b29, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/ab76f6080ca5446881a88d5804165b65] to archive 2023-03-27 03:55:29,099 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:29,100 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023032764d9cf497a8d4896a788b36c51b373ce_del 2023-03-27 03:55:29,100 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/2e74877b929c49099c6ed1b42da58dac to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/2e74877b929c49099c6ed1b42da58dac 2023-03-27 03:55:29,102 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/dc47b7d13f264c1f93acb1986c318032 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/dc47b7d13f264c1f93acb1986c318032 2023-03-27 03:55:29,103 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/43aa64ed29374656bbd548fe3dbccf48 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/43aa64ed29374656bbd548fe3dbccf48 2023-03-27 03:55:29,104 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/eae7a54eb7504ca0935439e5fc7a9b29 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/eae7a54eb7504ca0935439e5fc7a9b29 2023-03-27 03:55:29,104 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230327c76554099b7f4e9aa64f61a55973f42b_del 2023-03-27 03:55:29,105 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/ab76f6080ca5446881a88d5804165b65 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family2/ab76f6080ca5446881a88d5804165b65 2023-03-27 03:55:29,109 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230327b692662b8ecb4a02bd084a945e7bcceb_del 2023-03-27 03:55:29,109 INFO [Listener at localhost/36131] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-03-27 03:55:29,109 DEBUG [Listener at localhost/36131] zookeeper.ReadOnlyZKClient(139): Connect 0x534baca0 to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:55:29,115 DEBUG [Listener at localhost/36131] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3eae352b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:55:29,117 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230327 for table='ns:testMinorCompaction' and column='family1' 2023-03-27 03:55:29,117 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230327 for table='ns:testMinorCompaction' and column='family1' 2023-03-27 03:55:29,117 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230327 for table='ns:testMinorCompaction' and column='family1' 2023-03-27 03:55:29,194 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032799d305182b854c8a8d877ae15db55a90 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023032799d305182b854c8a8d877ae15db55a90 2023-03-27 03:55:29,195 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230327 table='ns:testMinorCompaction' column='family1' 2023-03-27 03:55:29,208 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032769120bfe9cdc44b4b38e7c11bca960f3 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023032769120bfe9cdc44b4b38e7c11bca960f3 2023-03-27 03:55:29,209 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230327 table='ns:testMinorCompaction' column='family1' 2023-03-27 03:55:29,212 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-27 03:55:29,217 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60868, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-27 03:55:29,239 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-27 03:55:29,252 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36186, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-27 03:55:29,287 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230327/family1/d929b93e5a29479abf61919990b16392 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-03-27 03:55:29,289 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230327/family1/ecd501197cfa41aead56239d9bf7343b first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-03-27 03:55:29,290 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230327/family1/d929b93e5a29479abf61919990b16392}] 2023-03-27 03:55:29,290 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230327/family1/ecd501197cfa41aead56239d9bf7343b}] 2023-03-27 03:55:29,300 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230327/family1/d929b93e5a29479abf61919990b16392 for inclusion in 9ce98af924f76608e57db6d1e9c69a22/family1 2023-03-27 03:55:29,301 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230327/family1/ecd501197cfa41aead56239d9bf7343b for inclusion in 72b74f7fa0bbf3322278037ab0d1d7f8/family1 2023-03-27 03:55:29,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-03-27 03:55:29,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-27 03:55:29,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for 9ce98af924f76608e57db6d1e9c69a22: 2023-03-27 03:55:29,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230327/family1/d929b93e5a29479abf61919990b16392 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__ns_testMinorCompaction__g3bqp263ddp2lk70jb6bg98k4idri6uhquse2u15o0d3csou1ikvj3oaicd2i7oe/family1/d929b93e5a29479abf61919990b16392 2023-03-27 03:55:29,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-03-27 03:55:29,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-27 03:55:29,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for 72b74f7fa0bbf3322278037ab0d1d7f8: 2023-03-27 03:55:29,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230327/family1/ecd501197cfa41aead56239d9bf7343b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__ns_testMinorCompaction__tuj5tgg1o717iu97p5nrelodto2bn92gaimrua75r3b9j0b80g2mkc8ipp4st9h1/family1/ecd501197cfa41aead56239d9bf7343b 2023-03-27 03:55:29,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__ns_testMinorCompaction__g3bqp263ddp2lk70jb6bg98k4idri6uhquse2u15o0d3csou1ikvj3oaicd2i7oe/family1/d929b93e5a29479abf61919990b16392 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/c1310903e3b3470b8545b3c897617c1d_SeqId_28_ 2023-03-27 03:55:29,308 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__ns_testMinorCompaction__g3bqp263ddp2lk70jb6bg98k4idri6uhquse2u15o0d3csou1ikvj3oaicd2i7oe/family1/d929b93e5a29479abf61919990b16392 into 9ce98af924f76608e57db6d1e9c69a22/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/c1310903e3b3470b8545b3c897617c1d_SeqId_28_ - updating store file list. 2023-03-27 03:55:29,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__ns_testMinorCompaction__tuj5tgg1o717iu97p5nrelodto2bn92gaimrua75r3b9j0b80g2mkc8ipp4st9h1/family1/ecd501197cfa41aead56239d9bf7343b as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/f2c731c0d10d4b268ef6df621642d62a_SeqId_28_ 2023-03-27 03:55:29,311 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/c1310903e3b3470b8545b3c897617c1d_SeqId_28_ into 9ce98af924f76608e57db6d1e9c69a22/family1 2023-03-27 03:55:29,311 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__ns_testMinorCompaction__tuj5tgg1o717iu97p5nrelodto2bn92gaimrua75r3b9j0b80g2mkc8ipp4st9h1/family1/ecd501197cfa41aead56239d9bf7343b into 72b74f7fa0bbf3322278037ab0d1d7f8/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/f2c731c0d10d4b268ef6df621642d62a_SeqId_28_ - updating store file list. 2023-03-27 03:55:29,311 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__ns_testMinorCompaction__g3bqp263ddp2lk70jb6bg98k4idri6uhquse2u15o0d3csou1ikvj3oaicd2i7oe/family1/d929b93e5a29479abf61919990b16392 into 9ce98af924f76608e57db6d1e9c69a22/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/family1/c1310903e3b3470b8545b3c897617c1d_SeqId_28_) 2023-03-27 03:55:29,312 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__ns_testMinorCompaction__g3bqp263ddp2lk70jb6bg98k4idri6uhquse2u15o0d3csou1ikvj3oaicd2i7oe/family1/d929b93e5a29479abf61919990b16392 2023-03-27 03:55:29,314 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230327 table='ns:testMinorCompaction' column='family1' 2023-03-27 03:55:29,315 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:29,317 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/f2c731c0d10d4b268ef6df621642d62a_SeqId_28_ into 72b74f7fa0bbf3322278037ab0d1d7f8/family1 2023-03-27 03:55:29,317 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__ns_testMinorCompaction__tuj5tgg1o717iu97p5nrelodto2bn92gaimrua75r3b9j0b80g2mkc8ipp4st9h1/family1/ecd501197cfa41aead56239d9bf7343b into 72b74f7fa0bbf3322278037ab0d1d7f8/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/family1/f2c731c0d10d4b268ef6df621642d62a_SeqId_28_) 2023-03-27 03:55:29,318 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303274280863728994b84bd4be9e6d467f8e7 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202303274280863728994b84bd4be9e6d467f8e7 2023-03-27 03:55:29,318 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-27 03:55:29,318 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230327' 2023-03-27 03:55:29,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__ns_testMinorCompaction__tuj5tgg1o717iu97p5nrelodto2bn92gaimrua75r3b9j0b80g2mkc8ipp4st9h1/family1/ecd501197cfa41aead56239d9bf7343b 2023-03-27 03:55:29,318 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230327 for table='ns:testMinorCompaction' and column='family1' 2023-03-27 03:55:29,321 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230327 table='ns:testMinorCompaction' column='family1' 2023-03-27 03:55:29,322 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:29,323 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230327a42afebf3aba43fbac870427faff1b88 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230327a42afebf3aba43fbac870427faff1b88 2023-03-27 03:55:29,323 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-27 03:55:29,323 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230327' 2023-03-27 03:55:29,323 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230327 for table='ns:testMinorCompaction' and column='family1' 2023-03-27 03:55:29,590 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230327f8cbe5880d884bc59da2f5a78b0cfd29 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230327f8cbe5880d884bc59da2f5a78b0cfd29 2023-03-27 03:55:29,591 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230327 table='ns:testMinorCompaction' column='family1' 2023-03-27 03:55:29,639 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230327/family1/58cb3e962e9c42089bdd923934cd1d81 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-03-27 03:55:29,639 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230327/family1/58cb3e962e9c42089bdd923934cd1d81}] 2023-03-27 03:55:29,641 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230327/family1/58cb3e962e9c42089bdd923934cd1d81 for inclusion in 7bab4d71becda434aedcb5da6cc18929/family1 2023-03-27 03:55:29,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-03-27 03:55:29,643 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last=1 2023-03-27 03:55:29,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for 7bab4d71becda434aedcb5da6cc18929: 2023-03-27 03:55:29,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230327/family1/58cb3e962e9c42089bdd923934cd1d81 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__ns_testMinorCompaction__roevqh4s4cqqsrv973ve9s8b88hu41o0s2p8ittvne4cllktbht606t9781eksrm/family1/58cb3e962e9c42089bdd923934cd1d81 2023-03-27 03:55:29,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__ns_testMinorCompaction__roevqh4s4cqqsrv973ve9s8b88hu41o0s2p8ittvne4cllktbht606t9781eksrm/family1/58cb3e962e9c42089bdd923934cd1d81 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/30eea5ef340a45a081c39f8025db14db_SeqId_28_ 2023-03-27 03:55:29,647 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__ns_testMinorCompaction__roevqh4s4cqqsrv973ve9s8b88hu41o0s2p8ittvne4cllktbht606t9781eksrm/family1/58cb3e962e9c42089bdd923934cd1d81 into 7bab4d71becda434aedcb5da6cc18929/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/30eea5ef340a45a081c39f8025db14db_SeqId_28_ - updating store file list. 2023-03-27 03:55:29,650 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/30eea5ef340a45a081c39f8025db14db_SeqId_28_ into 7bab4d71becda434aedcb5da6cc18929/family1 2023-03-27 03:55:29,650 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__ns_testMinorCompaction__roevqh4s4cqqsrv973ve9s8b88hu41o0s2p8ittvne4cllktbht606t9781eksrm/family1/58cb3e962e9c42089bdd923934cd1d81 into 7bab4d71becda434aedcb5da6cc18929/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/family1/30eea5ef340a45a081c39f8025db14db_SeqId_28_) 2023-03-27 03:55:29,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__ns_testMinorCompaction__roevqh4s4cqqsrv973ve9s8b88hu41o0s2p8ittvne4cllktbht606t9781eksrm/family1/58cb3e962e9c42089bdd923934cd1d81 2023-03-27 03:55:29,652 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230327 table='ns:testMinorCompaction' column='family1' 2023-03-27 03:55:29,652 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:29,653 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230327111fe37acbdb48668789203989e5883a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230327111fe37acbdb48668789203989e5883a 2023-03-27 03:55:29,653 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-03-27 03:55:29,653 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230327' 2023-03-27 03:55:29,654 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230327 for table='ns:testMinorCompaction' and column='family1' 2023-03-27 03:55:29,654 INFO [Listener at localhost/36131] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-03-27 03:55:29,656 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-27 03:55:29,658 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-27 03:55:29,672 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-27 03:55:29,673 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0f1f0589 to 127.0.0.1:62246 2023-03-27 03:55:29,673 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:55:29,673 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-27 03:55:29,674 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-27 03:55:29,674 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x534baca0 to 127.0.0.1:62246 2023-03-27 03:55:29,674 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:55:29,675 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-27 03:55:29,702 INFO [Listener at localhost/36131] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=247 (was 275), OpenFileDescriptor=646 (was 711), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=391 (was 390) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=8622 (was 8639) 2023-03-27 03:55:29,718 INFO [Listener at localhost/36131] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=247, OpenFileDescriptor=646, MaxFileDescriptor=60000, SystemLoadAverage=391, ProcessCount=169, AvailableMemoryMB=8621 2023-03-27 03:55:29,720 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-03-27 03:55:29,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:29,722 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-03-27 03:55:29,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-03-27 03:55:29,723 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-27 03:55:29,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-27 03:55:29,732 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489 2023-03-27 03:55:29,732 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489 empty. 2023-03-27 03:55:29,733 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489 2023-03-27 03:55:29,733 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-03-27 03:55:29,746 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-03-27 03:55:29,749 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 5f700223782e72e7df2f686db6ddd489, NAME => 'testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp 2023-03-27 03:55:29,756 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:55:29,756 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 5f700223782e72e7df2f686db6ddd489, disabling compactions & flushes 2023-03-27 03:55:29,756 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:29,756 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:29,756 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. after waiting 0 ms 2023-03-27 03:55:29,756 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:29,756 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:29,756 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 5f700223782e72e7df2f686db6ddd489: 2023-03-27 03:55:29,760 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-03-27 03:55:29,761 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679889329760"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889329760"}]},"ts":"1679889329760"} 2023-03-27 03:55:29,763 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-03-27 03:55:29,763 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-27 03:55:29,763 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889329763"}]},"ts":"1679889329763"} 2023-03-27 03:55:29,765 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-03-27 03:55:29,769 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=5f700223782e72e7df2f686db6ddd489, ASSIGN}] 2023-03-27 03:55:29,771 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=5f700223782e72e7df2f686db6ddd489, ASSIGN 2023-03-27 03:55:29,772 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=5f700223782e72e7df2f686db6ddd489, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=false 2023-03-27 03:55:29,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-27 03:55:29,923 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=5f700223782e72e7df2f686db6ddd489, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:29,923 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679889329923"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889329923"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889329923"}]},"ts":"1679889329923"} 2023-03-27 03:55:29,925 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure 5f700223782e72e7df2f686db6ddd489, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:55:30,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-27 03:55:30,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:30,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 5f700223782e72e7df2f686db6ddd489, NAME => 'testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489.', STARTKEY => '', ENDKEY => ''} 2023-03-27 03:55:30,080 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy 5f700223782e72e7df2f686db6ddd489 2023-03-27 03:55:30,080 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:55:30,080 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 5f700223782e72e7df2f686db6ddd489 2023-03-27 03:55:30,080 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 5f700223782e72e7df2f686db6ddd489 2023-03-27 03:55:30,081 INFO [StoreOpener-5f700223782e72e7df2f686db6ddd489-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 5f700223782e72e7df2f686db6ddd489 2023-03-27 03:55:30,083 DEBUG [StoreOpener-5f700223782e72e7df2f686db6ddd489-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1 2023-03-27 03:55:30,083 DEBUG [StoreOpener-5f700223782e72e7df2f686db6ddd489-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1 2023-03-27 03:55:30,083 INFO [StoreOpener-5f700223782e72e7df2f686db6ddd489-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5f700223782e72e7df2f686db6ddd489 columnFamilyName family1 2023-03-27 03:55:30,085 INFO [StoreOpener-5f700223782e72e7df2f686db6ddd489-1] regionserver.HStore(310): Store=5f700223782e72e7df2f686db6ddd489/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:55:30,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489 2023-03-27 03:55:30,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489 2023-03-27 03:55:30,088 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 5f700223782e72e7df2f686db6ddd489 2023-03-27 03:55:30,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:55:30,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 5f700223782e72e7df2f686db6ddd489; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67754324, jitterRate=0.009618103504180908}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-03-27 03:55:30,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 5f700223782e72e7df2f686db6ddd489: 2023-03-27 03:55:30,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489., pid=114, masterSystemTime=1679889330077 2023-03-27 03:55:30,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:30,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:30,093 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=5f700223782e72e7df2f686db6ddd489, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,093 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1679889330093"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889330093"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889330093"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889330093"}]},"ts":"1679889330093"} 2023-03-27 03:55:30,096 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-03-27 03:55:30,096 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure 5f700223782e72e7df2f686db6ddd489, server=jenkins-hbase4.apache.org,39727,1679889218560 in 169 msec 2023-03-27 03:55:30,097 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-03-27 03:55:30,097 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=5f700223782e72e7df2f686db6ddd489, ASSIGN in 327 msec 2023-03-27 03:55:30,097 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-27 03:55:30,097 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889330097"}]},"ts":"1679889330097"} 2023-03-27 03:55:30,098 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-03-27 03:55:30,101 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-03-27 03:55:30,102 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 381 msec 2023-03-27 03:55:30,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-03-27 03:55:30,327 INFO [Listener at localhost/36131] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-03-27 03:55:30,344 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:30,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,352 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-27 03:55:30,352 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:30,352 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-27 03:55:30,352 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-27 03:55:30,352 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,352 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,354 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,354 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:30,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:30,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:30,354 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,354 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:30,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,354 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:30,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,355 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,356 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,356 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-27 03:55:30,356 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:30,356 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-27 03:55:30,366 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:55:30,366 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:30,366 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:30,366 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. started... 2023-03-27 03:55:30,366 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5f700223782e72e7df2f686db6ddd489 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-03-27 03:55:30,392 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113037b3266a65ef4913b02b8080decc87ff to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113037b3266a65ef4913b02b8080decc87ff 2023-03-27 03:55:30,797 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/dcdb18bc18dc45688d14cd5260034375 2023-03-27 03:55:30,801 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/dcdb18bc18dc45688d14cd5260034375 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/dcdb18bc18dc45688d14cd5260034375 2023-03-27 03:55:30,805 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/dcdb18bc18dc45688d14cd5260034375, entries=1000, sequenceid=5, filesize=166.0 K 2023-03-27 03:55:30,806 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 5f700223782e72e7df2f686db6ddd489 in 440ms, sequenceid=5, compaction requested=false 2023-03-27 03:55:30,806 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-03-27 03:55:30,806 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5f700223782e72e7df2f686db6ddd489: 2023-03-27 03:55:30,806 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:30,806 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:55:30,806 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:55:30,806 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,806 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-27 03:55:30,806 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-27 03:55:30,808 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,808 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:30,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:30,808 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,809 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:30,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:30,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:30,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:30,810 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-27 03:55:30,810 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53c5c8fd[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:30,810 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-27 03:55:30,810 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,812 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,812 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-27 03:55:30,812 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-27 03:55:30,812 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:30,812 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,812 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:30,814 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-27 03:55:30,814 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:30,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:30,814 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:30,814 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-27 03:55:30,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:30,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:30,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:30,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:30,817 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:55:30,817 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-27 03:55:30,817 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:30,817 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:30,817 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,817 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:30,819 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,819 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:30,819 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:30,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:30,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:30,820 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:30,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:30,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:30,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,822 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:30,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,824 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,824 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:30,824 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:30,824 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,824 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,824 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,825 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,824 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-27 03:55:30,825 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-27 03:55:30,824 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:30,824 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:30,825 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:30,825 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:30,825 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:30,825 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:30,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:30,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:30,925 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:30,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:30,932 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:30,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:55:30,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,934 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-27 03:55:30,934 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:30,934 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-27 03:55:30,934 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-27 03:55:30,934 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,934 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,936 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,936 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:30,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:30,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:30,937 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,937 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:30,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:30,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,938 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-27 03:55:30,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,938 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-27 03:55:30,938 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:30,938 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-27 03:55:30,938 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:55:30,938 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:30,938 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:30,938 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. started... 2023-03-27 03:55:30,938 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5f700223782e72e7df2f686db6ddd489 1/1 column families, dataSize=74 B heapSize=408 B 2023-03-27 03:55:30,954 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128e2f26f75629e47e599a8cec68fc9cf6e to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128e2f26f75629e47e599a8cec68fc9cf6e 2023-03-27 03:55:30,960 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/61d8f3b1353f4a769dfb44adba3744e4 2023-03-27 03:55:30,963 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/61d8f3b1353f4a769dfb44adba3744e4 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/61d8f3b1353f4a769dfb44adba3744e4 2023-03-27 03:55:30,967 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/61d8f3b1353f4a769dfb44adba3744e4, entries=1, sequenceid=9, filesize=5.0 K 2023-03-27 03:55:30,967 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 5f700223782e72e7df2f686db6ddd489 in 29ms, sequenceid=9, compaction requested=false 2023-03-27 03:55:30,967 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5f700223782e72e7df2f686db6ddd489: 2023-03-27 03:55:30,967 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:30,968 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:55:30,968 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:55:30,968 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,968 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-27 03:55:30,968 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-27 03:55:30,969 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,969 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:30,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:30,970 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,970 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:30,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:30,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:30,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:30,971 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-27 03:55:30,971 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1bdf18ce[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:30,971 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-27 03:55:30,971 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,972 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,973 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-27 03:55:30,973 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-27 03:55:30,973 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:30,973 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,973 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:30,974 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,974 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-27 03:55:30,974 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:30,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:30,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:30,974 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-27 03:55:30,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:30,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:30,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:30,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:30,977 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:55:30,977 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-27 03:55:30,977 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:30,977 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:30,977 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,977 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:30,979 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,979 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:30,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:30,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,979 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:30,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:30,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:30,980 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:30,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:30,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,981 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:30,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,983 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,983 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,983 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,983 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:30,983 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-27 03:55:30,983 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:30,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:30,983 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:30,983 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:30,983 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:30,984 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:30,984 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,984 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:30,984 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:30,984 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:30,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:30,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:31,084 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:31,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:31,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:31,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:55:31,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,090 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-27 03:55:31,091 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:31,091 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-27 03:55:31,091 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-27 03:55:31,091 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,091 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,095 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,095 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:31,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:31,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:31,096 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,096 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:31,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,096 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:31,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,096 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,097 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-27 03:55:31,097 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,097 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-27 03:55:31,097 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:31,097 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-27 03:55:31,097 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:55:31,097 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:31,097 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:31,097 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. started... 2023-03-27 03:55:31,097 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5f700223782e72e7df2f686db6ddd489 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:55:31,114 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512018651ba03a5014cbca311f63acd2b2f7a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512018651ba03a5014cbca311f63acd2b2f7a 2023-03-27 03:55:31,520 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/ef663973f8344d85a323cd3bba5fb50c 2023-03-27 03:55:31,527 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/ef663973f8344d85a323cd3bba5fb50c as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/ef663973f8344d85a323cd3bba5fb50c 2023-03-27 03:55:31,531 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/ef663973f8344d85a323cd3bba5fb50c, entries=1, sequenceid=13, filesize=4.9 K 2023-03-27 03:55:31,532 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5f700223782e72e7df2f686db6ddd489 in 435ms, sequenceid=13, compaction requested=true 2023-03-27 03:55:31,532 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5f700223782e72e7df2f686db6ddd489: 2023-03-27 03:55:31,532 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:31,532 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:55:31,533 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:55:31,533 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,533 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-27 03:55:31,533 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-27 03:55:31,535 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,536 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:31,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:31,536 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,536 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:31,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:31,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:31,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:31,538 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-27 03:55:31,538 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6161dff3[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:31,538 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-27 03:55:31,538 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,539 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,539 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-27 03:55:31,539 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-27 03:55:31,539 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:31,539 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,540 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:31,541 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-27 03:55:31,541 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,541 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:31,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:31,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:31,541 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-27 03:55:31,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:31,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:31,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:31,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:31,550 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:55:31,550 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-27 03:55:31,550 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:31,550 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:31,550 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,550 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:31,552 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,552 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:31,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:31,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,552 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:31,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:31,553 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:31,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:31,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:31,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,554 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:31,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,559 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:31,559 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:31,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:31,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:31,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:31,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:31,563 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,563 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,563 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,563 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,563 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,563 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:31,563 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,563 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,563 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-27 03:55:31,563 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:31,563 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:31,563 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-27 03:55:31,563 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:31,664 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:31,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:31,669 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:31,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:55:31,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,670 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-27 03:55:31,670 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:31,671 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-27 03:55:31,671 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-27 03:55:31,671 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,671 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,672 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,672 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:31,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:31,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:31,673 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,673 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:31,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,673 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:31,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,674 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,674 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-27 03:55:31,674 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,674 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-27 03:55:31,674 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:31,674 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-27 03:55:31,674 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:55:31,675 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:31,675 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:31,675 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. started... 2023-03-27 03:55:31,675 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5f700223782e72e7df2f686db6ddd489 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:55:31,693 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512055348f98dc05b4a06863b1c46859f1806 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512055348f98dc05b4a06863b1c46859f1806 2023-03-27 03:55:31,715 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/018a403011d64d1db1567f857be8c8d4 2023-03-27 03:55:31,719 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/018a403011d64d1db1567f857be8c8d4 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/018a403011d64d1db1567f857be8c8d4 2023-03-27 03:55:31,724 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/018a403011d64d1db1567f857be8c8d4, entries=1, sequenceid=17, filesize=4.9 K 2023-03-27 03:55:31,724 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5f700223782e72e7df2f686db6ddd489 in 49ms, sequenceid=17, compaction requested=true 2023-03-27 03:55:31,725 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5f700223782e72e7df2f686db6ddd489: 2023-03-27 03:55:31,725 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:31,725 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:55:31,725 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:55:31,725 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,725 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-27 03:55:31,725 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-27 03:55:31,727 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,727 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:31,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:31,727 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,727 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:31,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:31,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:31,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:31,729 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-27 03:55:31,729 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@32f677f2[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:31,729 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-27 03:55:31,729 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,730 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,731 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-27 03:55:31,731 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-27 03:55:31,731 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:31,731 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,731 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:31,733 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,733 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-27 03:55:31,733 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:31,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:31,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:31,733 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-27 03:55:31,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:31,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:31,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:31,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:31,736 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:55:31,736 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-27 03:55:31,736 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:31,736 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:31,736 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,736 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:31,738 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,738 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,738 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:31,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:31,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:31,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,738 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:31,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:31,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:31,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:31,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,739 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:31,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,742 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:31,743 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:31,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:31,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:31,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:31,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:31,748 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,748 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,748 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,748 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,748 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,748 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:31,748 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-27 03:55:31,748 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:31,748 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,748 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,748 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-27 03:55:31,748 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:31,749 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:31,849 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:31,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:31,854 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:31,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:55:31,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,856 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-27 03:55:31,856 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:31,856 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-27 03:55:31,856 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-27 03:55:31,856 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,856 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,858 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:31,858 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:31,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:31,858 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:31,858 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:31,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,859 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:31,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,859 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,859 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-27 03:55:31,859 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:31,859 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-27 03:55:31,859 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:31,860 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-27 03:55:31,860 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:55:31,860 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:31,860 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:31,860 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. started... 2023-03-27 03:55:31,860 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5f700223782e72e7df2f686db6ddd489 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:55:31,884 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512288b554bf0cd9b470f9361b7ec7888595c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512288b554bf0cd9b470f9361b7ec7888595c 2023-03-27 03:55:32,291 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/e5ea15e2da3c4aaba3ed9c67c79ffe64 2023-03-27 03:55:32,296 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/e5ea15e2da3c4aaba3ed9c67c79ffe64 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/e5ea15e2da3c4aaba3ed9c67c79ffe64 2023-03-27 03:55:32,300 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/e5ea15e2da3c4aaba3ed9c67c79ffe64, entries=1, sequenceid=21, filesize=4.9 K 2023-03-27 03:55:32,301 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5f700223782e72e7df2f686db6ddd489 in 441ms, sequenceid=21, compaction requested=true 2023-03-27 03:55:32,301 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5f700223782e72e7df2f686db6ddd489: 2023-03-27 03:55:32,301 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:32,302 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:55:32,302 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:55:32,302 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,302 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-27 03:55:32,302 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-27 03:55:32,304 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,304 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:32,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:32,305 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,305 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:32,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:32,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:32,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:32,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-27 03:55:32,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4f09c371[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:32,307 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-27 03:55:32,307 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,308 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,308 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-27 03:55:32,308 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-27 03:55:32,308 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,308 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:32,308 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:32,310 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-27 03:55:32,310 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,310 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:32,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:32,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:32,310 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-27 03:55:32,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:32,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:32,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:32,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:32,313 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:55:32,313 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-27 03:55:32,313 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:32,313 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:32,313 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,313 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:32,315 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,315 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:32,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:32,315 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:32,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:32,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:32,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:32,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,317 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:32,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:32,318 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,326 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,326 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,326 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,326 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:32,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-27 03:55:32,327 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:32,326 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:32,326 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,327 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,327 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,327 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,326 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:32,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:32,327 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:32,327 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:32,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:32,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:32,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:32,427 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:32,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:32,432 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:32,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:55:32,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,434 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-27 03:55:32,434 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:32,434 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-27 03:55:32,434 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-27 03:55:32,435 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,435 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,437 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:32,437 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:32,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:32,438 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,438 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:32,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,438 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:32,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,438 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,446 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-27 03:55:32,446 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,446 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-27 03:55:32,447 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:32,447 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-27 03:55:32,447 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:55:32,447 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:32,447 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:32,447 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. started... 2023-03-27 03:55:32,447 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5f700223782e72e7df2f686db6ddd489 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:55:32,468 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512317849d1e639f44a61849b8cbd9e748943 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512317849d1e639f44a61849b8cbd9e748943 2023-03-27 03:55:32,481 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/efab5df772364e1ca855ff9ff883d5b6 2023-03-27 03:55:32,486 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/efab5df772364e1ca855ff9ff883d5b6 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/efab5df772364e1ca855ff9ff883d5b6 2023-03-27 03:55:32,492 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/efab5df772364e1ca855ff9ff883d5b6, entries=1, sequenceid=25, filesize=4.9 K 2023-03-27 03:55:32,494 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5f700223782e72e7df2f686db6ddd489 in 47ms, sequenceid=25, compaction requested=true 2023-03-27 03:55:32,494 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5f700223782e72e7df2f686db6ddd489: 2023-03-27 03:55:32,494 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:32,494 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:55:32,494 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:55:32,494 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,494 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-27 03:55:32,494 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-27 03:55:32,496 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,496 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:32,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:32,496 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,496 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:32,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:32,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:32,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:32,500 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-27 03:55:32,501 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2cf3ab36[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:32,501 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-27 03:55:32,501 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,502 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,502 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-27 03:55:32,502 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-27 03:55:32,502 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:32,502 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,502 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:32,503 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:32,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:32,503 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-27 03:55:32,504 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:32,504 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-27 03:55:32,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:32,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:32,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:32,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:32,506 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:55:32,506 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-27 03:55:32,506 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:32,507 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:32,507 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,507 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:32,509 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,509 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,509 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:32,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:32,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:32,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:32,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:32,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:32,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,511 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,512 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:32,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,515 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,515 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:32,515 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:32,515 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:32,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:32,515 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,515 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:32,515 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-27 03:55:32,515 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:32,515 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,515 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:32,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:32,515 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:32,515 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:32,516 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:32,515 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,517 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-27 03:55:32,616 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:32,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:32,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:32,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:55:32,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,622 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-27 03:55:32,622 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:32,622 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-27 03:55:32,622 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-27 03:55:32,622 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,622 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,624 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:32,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:32,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:32,624 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,624 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:32,624 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:32,624 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:32,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,625 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,625 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-27 03:55:32,625 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:32,625 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-27 03:55:32,625 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:32,625 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-27 03:55:32,625 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:55:32,625 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:32,625 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:32,625 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. started... 2023-03-27 03:55:32,625 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5f700223782e72e7df2f686db6ddd489 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:55:33,042 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010124b81b8c0bfd4a9c8eace729ba98c194 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010124b81b8c0bfd4a9c8eace729ba98c194 2023-03-27 03:55:33,449 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/8b7682d91c7849e7a0d6f158a5349ce5 2023-03-27 03:55:33,454 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/8b7682d91c7849e7a0d6f158a5349ce5 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/8b7682d91c7849e7a0d6f158a5349ce5 2023-03-27 03:55:33,458 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/8b7682d91c7849e7a0d6f158a5349ce5, entries=1, sequenceid=29, filesize=4.9 K 2023-03-27 03:55:33,459 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5f700223782e72e7df2f686db6ddd489 in 834ms, sequenceid=29, compaction requested=true 2023-03-27 03:55:33,459 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5f700223782e72e7df2f686db6ddd489: 2023-03-27 03:55:33,459 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:33,459 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:55:33,459 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:55:33,459 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,459 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-27 03:55:33,459 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-27 03:55:33,461 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,461 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:33,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:33,461 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:33,461 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:33,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:33,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:33,463 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-27 03:55:33,463 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4302771c[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:33,463 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-27 03:55:33,463 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,464 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,464 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-27 03:55:33,464 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-27 03:55:33,464 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,464 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:33,464 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:33,466 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,466 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-27 03:55:33,466 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:33,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:33,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:33,466 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-27 03:55:33,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:33,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:33,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:33,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:33,468 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:55:33,469 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-27 03:55:33,469 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:33,469 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:33,469 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,469 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:33,471 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,471 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,471 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:33,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:33,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:33,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:33,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:33,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:33,472 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:33,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,473 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:33,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,476 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,476 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,476 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,476 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,476 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-27 03:55:33,476 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:33,476 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,476 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,476 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:33,476 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:33,476 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:33,476 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:33,476 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,476 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:33,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:33,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:33,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:33,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:33,576 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:33,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:33,581 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:33,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:55:33,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,583 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-27 03:55:33,583 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:33,583 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-27 03:55:33,583 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-27 03:55:33,583 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,583 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,585 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:33,585 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:33,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:33,585 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,585 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:33,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,585 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:33,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,586 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,586 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-27 03:55:33,586 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,586 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-27 03:55:33,586 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:33,586 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-27 03:55:33,586 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:55:33,586 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:33,586 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:33,587 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. started... 2023-03-27 03:55:33,587 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5f700223782e72e7df2f686db6ddd489 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:55:33,601 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601032c43014629e649078d22ef349ff8b769 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601032c43014629e649078d22ef349ff8b769 2023-03-27 03:55:33,611 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/1bc7e00107fb4a32943ebdf4421d4455 2023-03-27 03:55:33,615 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/1bc7e00107fb4a32943ebdf4421d4455 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/1bc7e00107fb4a32943ebdf4421d4455 2023-03-27 03:55:33,619 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/1bc7e00107fb4a32943ebdf4421d4455, entries=1, sequenceid=33, filesize=4.9 K 2023-03-27 03:55:33,620 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5f700223782e72e7df2f686db6ddd489 in 33ms, sequenceid=33, compaction requested=true 2023-03-27 03:55:33,620 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5f700223782e72e7df2f686db6ddd489: 2023-03-27 03:55:33,620 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:33,620 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:55:33,620 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:55:33,620 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,620 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-27 03:55:33,620 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-27 03:55:33,622 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,622 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:33,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:33,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:33,622 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,622 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:33,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:33,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:33,623 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-27 03:55:33,623 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@424ad169[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:33,623 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-27 03:55:33,623 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,626 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,626 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-27 03:55:33,626 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-27 03:55:33,626 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:33,626 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,626 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:33,627 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,627 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-27 03:55:33,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:33,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:33,627 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:33,627 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-27 03:55:33,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:33,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:33,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:33,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:33,637 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:55:33,637 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-27 03:55:33,637 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:33,637 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:33,637 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,637 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:33,641 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,641 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,641 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:33,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:33,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:33,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:33,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:33,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:33,642 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:33,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,643 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:33,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,647 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,647 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,647 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,647 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,647 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,647 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:33,647 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-27 03:55:33,647 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:33,647 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,647 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:33,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:33,647 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-27 03:55:33,647 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:33,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:33,647 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,647 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:33,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:33,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:33,647 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:33,748 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:33,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:33,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:33,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:55:33,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,753 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-27 03:55:33,753 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:33,753 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-27 03:55:33,753 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-27 03:55:33,753 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,753 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,755 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,755 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:33,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:33,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:33,756 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:33,756 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:33,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,756 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:33,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,756 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,757 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-27 03:55:33,757 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:33,757 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-27 03:55:33,757 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:33,757 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-27 03:55:33,757 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:55:33,757 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:33,757 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:33,757 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. started... 2023-03-27 03:55:33,757 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5f700223782e72e7df2f686db6ddd489 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:55:34,168 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907cb319f0864c24419aa6965af702c48bc to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20150907cb319f0864c24419aa6965af702c48bc 2023-03-27 03:55:34,584 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/f284646572ca4ce5951e1e20aacd2307 2023-03-27 03:55:34,587 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/f284646572ca4ce5951e1e20aacd2307 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/f284646572ca4ce5951e1e20aacd2307 2023-03-27 03:55:34,590 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/f284646572ca4ce5951e1e20aacd2307, entries=1, sequenceid=37, filesize=4.9 K 2023-03-27 03:55:34,591 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5f700223782e72e7df2f686db6ddd489 in 834ms, sequenceid=37, compaction requested=true 2023-03-27 03:55:34,591 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5f700223782e72e7df2f686db6ddd489: 2023-03-27 03:55:34,591 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:34,591 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:55:34,591 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:55:34,591 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:34,591 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-27 03:55:34,591 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-27 03:55:34,593 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,593 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:34,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:34,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:34,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:34,593 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,593 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:34,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:34,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:34,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:34,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:34,594 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-27 03:55:34,594 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@43d9801a[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:34,594 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-27 03:55:34,594 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,595 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,595 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-27 03:55:34,595 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-27 03:55:34,595 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:34,595 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:34,596 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:34,597 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-27 03:55:34,597 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:34,597 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:34,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:34,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:34,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:34,597 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-27 03:55:34,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:34,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:34,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:34,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:34,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:34,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:34,599 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:55:34,599 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-27 03:55:34,599 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:34,599 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:34,599 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,599 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:34,602 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,602 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,602 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:34,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:34,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:34,602 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:34,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:34,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:34,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:34,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:34,603 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:34,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:34,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:34,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,605 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:34,605 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:34,605 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,605 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:34,605 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,605 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-27 03:55:34,605 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:34,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:34,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:34,605 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:34,605 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-27 03:55:34,605 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:34,605 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:34,605 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:34,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:34,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:34,605 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,605 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,606 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,705 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:34,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:34,710 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:34,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-03-27 03:55:34,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,711 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-03-27 03:55:34,711 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:34,711 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-03-27 03:55:34,711 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-03-27 03:55:34,712 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,712 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,713 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:34,713 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:34,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:34,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:34,713 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:34,713 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:34,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,714 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:34,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,714 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,714 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-03-27 03:55:34,714 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:34,714 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-03-27 03:55:34,714 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:34,714 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-03-27 03:55:34,714 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-03-27 03:55:34,714 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:34,714 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:34,714 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. started... 2023-03-27 03:55:34,715 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 5f700223782e72e7df2f686db6ddd489 1/1 column families, dataSize=73 B heapSize=408 B 2023-03-27 03:55:35,128 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120918d7739e51c4e3c919463eff793a4ff to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120918d7739e51c4e3c919463eff793a4ff 2023-03-27 03:55:35,546 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/e548549d9faa46d2a1aa758124eaeed5 2023-03-27 03:55:35,549 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/e548549d9faa46d2a1aa758124eaeed5 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/e548549d9faa46d2a1aa758124eaeed5 2023-03-27 03:55:35,552 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/e548549d9faa46d2a1aa758124eaeed5, entries=1, sequenceid=41, filesize=4.9 K 2023-03-27 03:55:35,552 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 5f700223782e72e7df2f686db6ddd489 in 837ms, sequenceid=41, compaction requested=true 2023-03-27 03:55:35,552 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 5f700223782e72e7df2f686db6ddd489: 2023-03-27 03:55:35,552 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:35,553 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-03-27 03:55:35,553 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-03-27 03:55:35,553 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:35,553 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-03-27 03:55:35,553 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-03-27 03:55:35,554 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:35,554 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:35,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:35,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:35,555 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,555 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:35,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:35,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:35,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:35,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:35,556 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-03-27 03:55:35,556 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72c2294c[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:35,556 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-03-27 03:55:35,556 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,557 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,557 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-03-27 03:55:35,557 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-03-27 03:55:35,557 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:35,557 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:35,557 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:35,559 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-03-27 03:55:35,559 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:35,559 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:35,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:35,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:35,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:35,559 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-03-27 03:55:35,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:35,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:35,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:35,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:35,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:35,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:35,562 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-03-27 03:55:35,562 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-03-27 03:55:35,562 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:35,562 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:35,562 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,562 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:35,563 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,563 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:35,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:35,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,563 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:35,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:35,564 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:35,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:35,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:35,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:35,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,564 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:35,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:35,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:35,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:35,566 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:35,567 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:35,567 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:35,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:35,567 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:35,567 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-03-27 03:55:35,567 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:35,567 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:35,567 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:35,567 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:35,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:35,567 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,567 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-03-27 03:55:35,567 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:35,567 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,567 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:35,567 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,626 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-27 03:55:35,667 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:35,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:35,669 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38271] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-03-27 03:55:35,670 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-27 03:55:35,671 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-27 03:55:35,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:55:35,672 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679889935672 2023-03-27 03:55:35,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-03-27 03:55:35,673 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-03-27 03:55:35,673 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-27 03:55:35,673 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] zookeeper.ReadOnlyZKClient(139): Connect 0x425b86ea to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:55:35,678 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7040141a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:55:35,678 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-27 03:55:36,500 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103a0bba05641cf4e7cb0005e0bca601e71 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103a0bba05641cf4e7cb0005e0bca601e71 2023-03-27 03:55:36,501 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-27 03:55:36,513 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-27 03:55:36,515 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42522, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-27 03:55:36,523 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-27 03:55:36,524 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45032, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-27 03:55:36,532 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/c17509e478d940e68e4e90beae737d58 first=Optional[r5] last=Optional[r6] 2023-03-27 03:55:36,533 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/c17509e478d940e68e4e90beae737d58}] 2023-03-27 03:55:36,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/c17509e478d940e68e4e90beae737d58 for inclusion in 5f700223782e72e7df2f686db6ddd489/family1 2023-03-27 03:55:36,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-03-27 03:55:36,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last= 2023-03-27 03:55:36,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for 5f700223782e72e7df2f686db6ddd489: 2023-03-27 03:55:36,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/c17509e478d940e68e4e90beae737d58 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithMonthlyPolicy__grag4veskmqt0o4tgqp2ud0djijeuc7fp3clci5sd6u28sqauf3ks34vqhrlvug8/family1/c17509e478d940e68e4e90beae737d58 2023-03-27 03:55:36,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithMonthlyPolicy__grag4veskmqt0o4tgqp2ud0djijeuc7fp3clci5sd6u28sqauf3ks34vqhrlvug8/family1/c17509e478d940e68e4e90beae737d58 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/8c53ae11c5d14e189aa1079887aac091_SeqId_44_ 2023-03-27 03:55:36,542 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithMonthlyPolicy__grag4veskmqt0o4tgqp2ud0djijeuc7fp3clci5sd6u28sqauf3ks34vqhrlvug8/family1/c17509e478d940e68e4e90beae737d58 into 5f700223782e72e7df2f686db6ddd489/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/8c53ae11c5d14e189aa1079887aac091_SeqId_44_ - updating store file list. 2023-03-27 03:55:36,545 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/8c53ae11c5d14e189aa1079887aac091_SeqId_44_ into 5f700223782e72e7df2f686db6ddd489/family1 2023-03-27 03:55:36,545 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithMonthlyPolicy__grag4veskmqt0o4tgqp2ud0djijeuc7fp3clci5sd6u28sqauf3ks34vqhrlvug8/family1/c17509e478d940e68e4e90beae737d58 into 5f700223782e72e7df2f686db6ddd489/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/8c53ae11c5d14e189aa1079887aac091_SeqId_44_) 2023-03-27 03:55:36,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithMonthlyPolicy__grag4veskmqt0o4tgqp2ud0djijeuc7fp3clci5sd6u28sqauf3ks34vqhrlvug8/family1/c17509e478d940e68e4e90beae737d58 2023-03-27 03:55:36,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-03-27 03:55:36,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-03-27 03:55:36,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(1912): 5f700223782e72e7df2f686db6ddd489/family1 is initiating minor compaction 2023-03-27 03:55:36,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@197ef04; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:55:36,548 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 5f700223782e72e7df2f686db6ddd489/family1 in testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:36,548 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/61d8f3b1353f4a769dfb44adba3744e4, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/ef663973f8344d85a323cd3bba5fb50c, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/018a403011d64d1db1567f857be8c8d4, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/e5ea15e2da3c4aaba3ed9c67c79ffe64, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/efab5df772364e1ca855ff9ff883d5b6, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/8b7682d91c7849e7a0d6f158a5349ce5, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/1bc7e00107fb4a32943ebdf4421d4455, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/f284646572ca4ce5951e1e20aacd2307, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/e548549d9faa46d2a1aa758124eaeed5, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/8c53ae11c5d14e189aa1079887aac091_SeqId_44_] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp, totalSize=49.7 K 2023-03-27 03:55:36,548 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 61d8f3b1353f4a769dfb44adba3744e4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-03-27 03:55:36,548 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting ef663973f8344d85a323cd3bba5fb50c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-03-27 03:55:36,548 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 018a403011d64d1db1567f857be8c8d4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-03-27 03:55:36,548 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting e5ea15e2da3c4aaba3ed9c67c79ffe64, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-03-27 03:55:36,548 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting efab5df772364e1ca855ff9ff883d5b6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-03-27 03:55:36,548 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 8b7682d91c7849e7a0d6f158a5349ce5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-03-27 03:55:36,548 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 1bc7e00107fb4a32943ebdf4421d4455, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-03-27 03:55:36,548 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting f284646572ca4ce5951e1e20aacd2307, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-03-27 03:55:36,548 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting e548549d9faa46d2a1aa758124eaeed5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-03-27 03:55:36,548 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 8c53ae11c5d14e189aa1079887aac091_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-03-27 03:55:36,549 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-27 03:55:36,550 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:36,551 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010124b81b8c0bfd4a9c8eace729ba98c194 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010124b81b8c0bfd4a9c8eace729ba98c194 2023-03-27 03:55:36,553 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601032c43014629e649078d22ef349ff8b769 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601032c43014629e649078d22ef349ff8b769 2023-03-27 03:55:36,553 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-03-27 03:55:36,553 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-03-27 03:55:36,553 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-27 03:55:36,553 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-27 03:55:36,577 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5f700223782e72e7df2f686db6ddd489#family1#compaction#166 average throughput is 1.37 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:37,389 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/87cceadf156141e493141b55396bd5b6 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/87cceadf156141e493141b55396bd5b6 2023-03-27 03:55:37,392 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231d392043dd0aa476db346e66294796b4f to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231d392043dd0aa476db346e66294796b4f 2023-03-27 03:55:37,393 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-27 03:55:37,394 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 5f700223782e72e7df2f686db6ddd489/family1 of 5f700223782e72e7df2f686db6ddd489 into 87cceadf156141e493141b55396bd5b6(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:55:37,394 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 5f700223782e72e7df2f686db6ddd489: 2023-03-27 03:55:37,394 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1679889336547; duration=0sec 2023-03-27 03:55:37,394 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:55:37,420 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/307cd47782374c76a863adb7b112a92d first=Optional[r1] last=Optional[r4] 2023-03-27 03:55:37,421 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/307cd47782374c76a863adb7b112a92d}] 2023-03-27 03:55:37,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/307cd47782374c76a863adb7b112a92d for inclusion in 5f700223782e72e7df2f686db6ddd489/family1 2023-03-27 03:55:37,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-03-27 03:55:37,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last= 2023-03-27 03:55:37,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for 5f700223782e72e7df2f686db6ddd489: 2023-03-27 03:55:37,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/307cd47782374c76a863adb7b112a92d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithMonthlyPolicy__rc5osiak5ptbt16speeiunpkt34gbj8kamoi67nal914nhbtcc2umkrmtopi23sg/family1/307cd47782374c76a863adb7b112a92d 2023-03-27 03:55:37,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithMonthlyPolicy__rc5osiak5ptbt16speeiunpkt34gbj8kamoi67nal914nhbtcc2umkrmtopi23sg/family1/307cd47782374c76a863adb7b112a92d as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/e3285c9157d54c31af115f2b3229ece5_SeqId_47_ 2023-03-27 03:55:37,428 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithMonthlyPolicy__rc5osiak5ptbt16speeiunpkt34gbj8kamoi67nal914nhbtcc2umkrmtopi23sg/family1/307cd47782374c76a863adb7b112a92d into 5f700223782e72e7df2f686db6ddd489/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/e3285c9157d54c31af115f2b3229ece5_SeqId_47_ - updating store file list. 2023-03-27 03:55:37,430 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/e3285c9157d54c31af115f2b3229ece5_SeqId_47_ into 5f700223782e72e7df2f686db6ddd489/family1 2023-03-27 03:55:37,430 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithMonthlyPolicy__rc5osiak5ptbt16speeiunpkt34gbj8kamoi67nal914nhbtcc2umkrmtopi23sg/family1/307cd47782374c76a863adb7b112a92d into 5f700223782e72e7df2f686db6ddd489/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/e3285c9157d54c31af115f2b3229ece5_SeqId_47_) 2023-03-27 03:55:37,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithMonthlyPolicy__rc5osiak5ptbt16speeiunpkt34gbj8kamoi67nal914nhbtcc2umkrmtopi23sg/family1/307cd47782374c76a863adb7b112a92d 2023-03-27 03:55:37,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-03-27 03:55:37,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-03-27 03:55:37,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(1912): 5f700223782e72e7df2f686db6ddd489/family1 is initiating minor compaction (all files) 2023-03-27 03:55:37,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2123bdd6; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:55:37,432 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 5f700223782e72e7df2f686db6ddd489/family1 in testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:37,432 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/dcdb18bc18dc45688d14cd5260034375, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/87cceadf156141e493141b55396bd5b6, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/e3285c9157d54c31af115f2b3229ece5_SeqId_47_] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp, totalSize=178.1 K 2023-03-27 03:55:37,432 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting dcdb18bc18dc45688d14cd5260034375, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878842162 2023-03-27 03:55:37,432 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting 87cceadf156141e493141b55396bd5b6, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621242162 2023-03-27 03:55:37,433 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] compactions.Compactor(207): Compacting e3285c9157d54c31af115f2b3229ece5_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965242162 2023-03-27 03:55:37,433 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-27 03:55:37,434 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:37,435 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512018651ba03a5014cbca311f63acd2b2f7a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512018651ba03a5014cbca311f63acd2b2f7a 2023-03-27 03:55:37,436 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512055348f98dc05b4a06863b1c46859f1806 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512055348f98dc05b4a06863b1c46859f1806 2023-03-27 03:55:37,437 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512288b554bf0cd9b470f9361b7ec7888595c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512288b554bf0cd9b470f9361b7ec7888595c 2023-03-27 03:55:37,438 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512317849d1e639f44a61849b8cbd9e748943 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512317849d1e639f44a61849b8cbd9e748943 2023-03-27 03:55:37,438 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-03-27 03:55:37,438 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-03-27 03:55:37,438 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-27 03:55:37,438 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-27 03:55:37,446 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5f700223782e72e7df2f686db6ddd489#family1#compaction#167 average throughput is 22.23 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:37,498 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/61d8f3b1353f4a769dfb44adba3744e4, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/ef663973f8344d85a323cd3bba5fb50c, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/018a403011d64d1db1567f857be8c8d4, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/e5ea15e2da3c4aaba3ed9c67c79ffe64, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/efab5df772364e1ca855ff9ff883d5b6, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/8b7682d91c7849e7a0d6f158a5349ce5, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/1bc7e00107fb4a32943ebdf4421d4455, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/f284646572ca4ce5951e1e20aacd2307, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/e548549d9faa46d2a1aa758124eaeed5, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/8c53ae11c5d14e189aa1079887aac091_SeqId_44_] to archive 2023-03-27 03:55:37,499 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:37,500 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/61d8f3b1353f4a769dfb44adba3744e4 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/61d8f3b1353f4a769dfb44adba3744e4 2023-03-27 03:55:37,501 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/ef663973f8344d85a323cd3bba5fb50c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/ef663973f8344d85a323cd3bba5fb50c 2023-03-27 03:55:37,502 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/018a403011d64d1db1567f857be8c8d4 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/018a403011d64d1db1567f857be8c8d4 2023-03-27 03:55:37,503 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/e5ea15e2da3c4aaba3ed9c67c79ffe64 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/e5ea15e2da3c4aaba3ed9c67c79ffe64 2023-03-27 03:55:37,504 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/efab5df772364e1ca855ff9ff883d5b6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/efab5df772364e1ca855ff9ff883d5b6 2023-03-27 03:55:37,505 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/8b7682d91c7849e7a0d6f158a5349ce5 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/8b7682d91c7849e7a0d6f158a5349ce5 2023-03-27 03:55:37,506 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/1bc7e00107fb4a32943ebdf4421d4455 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/1bc7e00107fb4a32943ebdf4421d4455 2023-03-27 03:55:37,507 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/f284646572ca4ce5951e1e20aacd2307 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/f284646572ca4ce5951e1e20aacd2307 2023-03-27 03:55:37,508 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/e548549d9faa46d2a1aa758124eaeed5 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/e548549d9faa46d2a1aa758124eaeed5 2023-03-27 03:55:37,508 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/8c53ae11c5d14e189aa1079887aac091_SeqId_44_ to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/8c53ae11c5d14e189aa1079887aac091_SeqId_44_ 2023-03-27 03:55:38,270 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113082a9de5bb95446ee9a195badb91ce79b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113082a9de5bb95446ee9a195badb91ce79b 2023-03-27 03:55:38,271 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-27 03:55:38,295 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/e92acc8a67644f649f5e73d5e17c2a1d first=Optional[r00] last=Optional[r8] 2023-03-27 03:55:38,296 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/e92acc8a67644f649f5e73d5e17c2a1d}] 2023-03-27 03:55:38,297 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/e92acc8a67644f649f5e73d5e17c2a1d for inclusion in 5f700223782e72e7df2f686db6ddd489/family1 2023-03-27 03:55:38,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-03-27 03:55:38,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last= 2023-03-27 03:55:38,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for 5f700223782e72e7df2f686db6ddd489: 2023-03-27 03:55:38,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/e92acc8a67644f649f5e73d5e17c2a1d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithMonthlyPolicy__73drer7g5ulvsso9kurcjj06p9kc3au39stakei65cd35hk18llr060ur8hcqsda/family1/e92acc8a67644f649f5e73d5e17c2a1d 2023-03-27 03:55:38,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithMonthlyPolicy__73drer7g5ulvsso9kurcjj06p9kc3au39stakei65cd35hk18llr060ur8hcqsda/family1/e92acc8a67644f649f5e73d5e17c2a1d as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/8283781d7699481ab2f9985b956989fe_SeqId_49_ 2023-03-27 03:55:38,302 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithMonthlyPolicy__73drer7g5ulvsso9kurcjj06p9kc3au39stakei65cd35hk18llr060ur8hcqsda/family1/e92acc8a67644f649f5e73d5e17c2a1d into 5f700223782e72e7df2f686db6ddd489/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/8283781d7699481ab2f9985b956989fe_SeqId_49_ - updating store file list. 2023-03-27 03:55:38,305 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/8283781d7699481ab2f9985b956989fe_SeqId_49_ into 5f700223782e72e7df2f686db6ddd489/family1 2023-03-27 03:55:38,305 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithMonthlyPolicy__73drer7g5ulvsso9kurcjj06p9kc3au39stakei65cd35hk18llr060ur8hcqsda/family1/e92acc8a67644f649f5e73d5e17c2a1d into 5f700223782e72e7df2f686db6ddd489/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/8283781d7699481ab2f9985b956989fe_SeqId_49_) 2023-03-27 03:55:38,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMinorCompactionWithMonthlyPolicy__73drer7g5ulvsso9kurcjj06p9kc3au39stakei65cd35hk18llr060ur8hcqsda/family1/e92acc8a67644f649f5e73d5e17c2a1d 2023-03-27 03:55:38,307 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-27 03:55:38,308 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:38,309 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120918d7739e51c4e3c919463eff793a4ff to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120918d7739e51c4e3c919463eff793a4ff 2023-03-27 03:55:38,310 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128e2f26f75629e47e599a8cec68fc9cf6e to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128e2f26f75629e47e599a8cec68fc9cf6e 2023-03-27 03:55:38,311 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113037b3266a65ef4913b02b8080decc87ff to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113037b3266a65ef4913b02b8080decc87ff 2023-03-27 03:55:38,311 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-03-27 03:55:38,311 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-03-27 03:55:38,312 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-27 03:55:38,312 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-27 03:55:38,312 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-03-27 03:55:38,312 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-03-27 03:55:38,312 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:55:38,314 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-03-27 03:55:38,315 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.6440 sec 2023-03-27 03:55:38,491 INFO [Listener at localhost/36131] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=255 (was 247) - Thread LEAK? -, OpenFileDescriptor=642 (was 646), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=391 (was 391), ProcessCount=169 (was 169), AvailableMemoryMB=8614 (was 8621) 2023-03-27 03:55:38,503 INFO [Listener at localhost/36131] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=255, OpenFileDescriptor=642, MaxFileDescriptor=60000, SystemLoadAverage=391, ProcessCount=169, AvailableMemoryMB=8613 2023-03-27 03:55:38,522 DEBUG [Listener at localhost/36131] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-03-27 03:55:38,575 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCB7\xB8\xC1\xDF\xD3\xE90\xD2J\xC9\xD8lDb\xF3"\x10\x0D9,\xD6\xC2+]HP%\xFC2:\xDF<6*\x106\x85\xD2\xB4\xD2\xB2{\xD8\x026\xF5\xC6Q.e.2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-03-27 03:55:38,575 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-03-27 03:55:38,577 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-03-27 03:55:38,577 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-03-27 03:55:38,577 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-03-27 03:55:38,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-27 03:55:38,579 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6 2023-03-27 03:55:38,579 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a 2023-03-27 03:55:38,579 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9 2023-03-27 03:55:38,580 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6 empty. 2023-03-27 03:55:38,580 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9 empty. 2023-03-27 03:55:38,580 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a empty. 2023-03-27 03:55:38,580 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6 2023-03-27 03:55:38,580 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a 2023-03-27 03:55:38,580 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9 2023-03-27 03:55:38,580 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-03-27 03:55:38,663 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/.tmp/family1/f49279a58fed4ee6a5eeb907fc85c5ab as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/f49279a58fed4ee6a5eeb907fc85c5ab 2023-03-27 03:55:38,667 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 5f700223782e72e7df2f686db6ddd489/family1 of 5f700223782e72e7df2f686db6ddd489 into f49279a58fed4ee6a5eeb907fc85c5ab(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-27 03:55:38,667 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 5f700223782e72e7df2f686db6ddd489: 2023-03-27 03:55:38,667 INFO [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1679889337431; duration=1sec 2023-03-27 03:55:38,667 DEBUG [RS:0;jenkins-hbase4:39727-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-03-27 03:55:38,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-27 03:55:38,689 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/dcdb18bc18dc45688d14cd5260034375, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/87cceadf156141e493141b55396bd5b6, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/e3285c9157d54c31af115f2b3229ece5_SeqId_47_] to archive 2023-03-27 03:55:38,689 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:38,691 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/dcdb18bc18dc45688d14cd5260034375 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/dcdb18bc18dc45688d14cd5260034375 2023-03-27 03:55:38,692 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/87cceadf156141e493141b55396bd5b6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/87cceadf156141e493141b55396bd5b6 2023-03-27 03:55:38,693 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/e3285c9157d54c31af115f2b3229ece5_SeqId_47_ to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/family1/e3285c9157d54c31af115f2b3229ece5_SeqId_47_ 2023-03-27 03:55:38,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-27 03:55:38,987 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-03-27 03:55:38,988 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7675): creating {ENCODED => 95b223459399eec298605c6ce5cd65f6, NAME => 'testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCB7\xB8\xC1\xDF\xD3\xE90\xD2J\xC9\xD8lDb\xF3"\x10\x0D9,\xD6\xC2+]HP%\xFC2:\xDF<6*\x106\x85\xD2\xB4\xD2\xB2{\xD8\x026\xF5\xC6Q.e.2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp 2023-03-27 03:55:38,988 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7675): creating {ENCODED => d0bf0165da69488ffc28ab9679dbd8e9, NAME => 'testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCB7\xB8\xC1\xDF\xD3\xE90\xD2J\xC9\xD8lDb\xF3"\x10\x0D9,\xD6\xC2+]HP%\xFC2:\xDF<6*\x106\x85\xD2\xB4\xD2\xB2{\xD8\x026\xF5\xC6Q.e.2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp 2023-03-27 03:55:38,988 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7675): creating {ENCODED => ae7919e6ac686a9f3a2be09b01a0ff1a, NAME => 'testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCB7\xB8\xC1\xDF\xD3\xE90\xD2J\xC9\xD8lDb\xF3"\x10\x0D9,\xD6\xC2+]HP%\xFC2:\xDF<6*\x106\x85\xD2\xB4\xD2\xB2{\xD8\x026\xF5\xC6Q.e.2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/.tmp 2023-03-27 03:55:39,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-27 03:55:39,394 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:55:39,394 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1604): Closing ae7919e6ac686a9f3a2be09b01a0ff1a, disabling compactions & flushes 2023-03-27 03:55:39,394 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:39,394 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:39,394 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. after waiting 0 ms 2023-03-27 03:55:39,394 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:39,394 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:55:39,394 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:39,394 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1604): Closing d0bf0165da69488ffc28ab9679dbd8e9, disabling compactions & flushes 2023-03-27 03:55:39,394 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1558): Region close journal for ae7919e6ac686a9f3a2be09b01a0ff1a: 2023-03-27 03:55:39,394 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:39,394 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:39,394 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:55:39,394 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. after waiting 0 ms 2023-03-27 03:55:39,394 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1604): Closing 95b223459399eec298605c6ce5cd65f6, disabling compactions & flushes 2023-03-27 03:55:39,394 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:39,394 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:39,395 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:39,394 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:39,395 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1558): Region close journal for d0bf0165da69488ffc28ab9679dbd8e9: 2023-03-27 03:55:39,395 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. after waiting 0 ms 2023-03-27 03:55:39,395 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:39,395 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:39,395 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1558): Region close journal for 95b223459399eec298605c6ce5cd65f6: 2023-03-27 03:55:39,396 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-03-27 03:55:39,397 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1679889339397"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889339397"}]},"ts":"1679889339397"} 2023-03-27 03:55:39,397 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679889339397"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889339397"}]},"ts":"1679889339397"} 2023-03-27 03:55:39,397 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679889339397"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1679889339397"}]},"ts":"1679889339397"} 2023-03-27 03:55:39,399 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-03-27 03:55:39,400 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-03-27 03:55:39,400 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889339400"}]},"ts":"1679889339400"} 2023-03-27 03:55:39,401 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-03-27 03:55:39,403 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=95b223459399eec298605c6ce5cd65f6, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ae7919e6ac686a9f3a2be09b01a0ff1a, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=d0bf0165da69488ffc28ab9679dbd8e9, ASSIGN}] 2023-03-27 03:55:39,405 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=95b223459399eec298605c6ce5cd65f6, ASSIGN 2023-03-27 03:55:39,405 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=d0bf0165da69488ffc28ab9679dbd8e9, ASSIGN 2023-03-27 03:55:39,405 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ae7919e6ac686a9f3a2be09b01a0ff1a, ASSIGN 2023-03-27 03:55:39,405 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=95b223459399eec298605c6ce5cd65f6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=false 2023-03-27 03:55:39,405 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=d0bf0165da69488ffc28ab9679dbd8e9, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=false 2023-03-27 03:55:39,405 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ae7919e6ac686a9f3a2be09b01a0ff1a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39727,1679889218560; forceNewPlan=false, retain=false 2023-03-27 03:55:39,557 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=95b223459399eec298605c6ce5cd65f6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:39,557 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679889339557"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889339557"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889339557"}]},"ts":"1679889339557"} 2023-03-27 03:55:39,558 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=ae7919e6ac686a9f3a2be09b01a0ff1a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:39,558 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=d0bf0165da69488ffc28ab9679dbd8e9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:39,558 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1679889339558"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889339558"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889339558"}]},"ts":"1679889339558"} 2023-03-27 03:55:39,558 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679889339558"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1679889339558"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1679889339558"}]},"ts":"1679889339558"} 2023-03-27 03:55:39,559 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=117, state=RUNNABLE; OpenRegionProcedure 95b223459399eec298605c6ce5cd65f6, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:55:39,560 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=118, state=RUNNABLE; OpenRegionProcedure ae7919e6ac686a9f3a2be09b01a0ff1a, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:55:39,560 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=119, state=RUNNABLE; OpenRegionProcedure d0bf0165da69488ffc28ab9679dbd8e9, server=jenkins-hbase4.apache.org,39727,1679889218560}] 2023-03-27 03:55:39,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-27 03:55:39,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:39,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ae7919e6ac686a9f3a2be09b01a0ff1a, NAME => 'testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a.', STARTKEY => '1', ENDKEY => '2'} 2023-03-27 03:55:39,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin ae7919e6ac686a9f3a2be09b01a0ff1a 2023-03-27 03:55:39,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:55:39,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ae7919e6ac686a9f3a2be09b01a0ff1a 2023-03-27 03:55:39,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ae7919e6ac686a9f3a2be09b01a0ff1a 2023-03-27 03:55:39,745 INFO [StoreOpener-ae7919e6ac686a9f3a2be09b01a0ff1a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region ae7919e6ac686a9f3a2be09b01a0ff1a 2023-03-27 03:55:39,746 DEBUG [StoreOpener-ae7919e6ac686a9f3a2be09b01a0ff1a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1 2023-03-27 03:55:39,746 DEBUG [StoreOpener-ae7919e6ac686a9f3a2be09b01a0ff1a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1 2023-03-27 03:55:39,747 INFO [StoreOpener-ae7919e6ac686a9f3a2be09b01a0ff1a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ae7919e6ac686a9f3a2be09b01a0ff1a columnFamilyName family1 2023-03-27 03:55:39,748 INFO [StoreOpener-ae7919e6ac686a9f3a2be09b01a0ff1a-1] regionserver.HStore(310): Store=ae7919e6ac686a9f3a2be09b01a0ff1a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:55:39,748 INFO [StoreOpener-ae7919e6ac686a9f3a2be09b01a0ff1a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region ae7919e6ac686a9f3a2be09b01a0ff1a 2023-03-27 03:55:39,750 DEBUG [StoreOpener-ae7919e6ac686a9f3a2be09b01a0ff1a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2 2023-03-27 03:55:39,750 DEBUG [StoreOpener-ae7919e6ac686a9f3a2be09b01a0ff1a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2 2023-03-27 03:55:39,751 INFO [StoreOpener-ae7919e6ac686a9f3a2be09b01a0ff1a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ae7919e6ac686a9f3a2be09b01a0ff1a columnFamilyName family2 2023-03-27 03:55:39,752 INFO [StoreOpener-ae7919e6ac686a9f3a2be09b01a0ff1a-1] regionserver.HStore(310): Store=ae7919e6ac686a9f3a2be09b01a0ff1a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:55:39,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a 2023-03-27 03:55:39,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a 2023-03-27 03:55:39,754 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-27 03:55:39,755 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ae7919e6ac686a9f3a2be09b01a0ff1a 2023-03-27 03:55:39,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:55:39,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ae7919e6ac686a9f3a2be09b01a0ff1a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60788003, jitterRate=-0.09418816864490509}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-27 03:55:39,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ae7919e6ac686a9f3a2be09b01a0ff1a: 2023-03-27 03:55:39,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a., pid=121, masterSystemTime=1679889339711 2023-03-27 03:55:39,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:39,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:39,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:39,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 95b223459399eec298605c6ce5cd65f6, NAME => 'testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6.', STARTKEY => '', ENDKEY => '1'} 2023-03-27 03:55:39,763 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=ae7919e6ac686a9f3a2be09b01a0ff1a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:39,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 95b223459399eec298605c6ce5cd65f6 2023-03-27 03:55:39,763 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1679889339763"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889339763"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889339763"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889339763"}]},"ts":"1679889339763"} 2023-03-27 03:55:39,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:55:39,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 95b223459399eec298605c6ce5cd65f6 2023-03-27 03:55:39,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 95b223459399eec298605c6ce5cd65f6 2023-03-27 03:55:39,766 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=118 2023-03-27 03:55:39,766 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=118, state=SUCCESS; OpenRegionProcedure ae7919e6ac686a9f3a2be09b01a0ff1a, server=jenkins-hbase4.apache.org,39727,1679889218560 in 204 msec 2023-03-27 03:55:39,768 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=ae7919e6ac686a9f3a2be09b01a0ff1a, ASSIGN in 363 msec 2023-03-27 03:55:39,786 INFO [StoreOpener-95b223459399eec298605c6ce5cd65f6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 95b223459399eec298605c6ce5cd65f6 2023-03-27 03:55:39,787 DEBUG [StoreOpener-95b223459399eec298605c6ce5cd65f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1 2023-03-27 03:55:39,788 DEBUG [StoreOpener-95b223459399eec298605c6ce5cd65f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1 2023-03-27 03:55:39,788 INFO [StoreOpener-95b223459399eec298605c6ce5cd65f6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 95b223459399eec298605c6ce5cd65f6 columnFamilyName family1 2023-03-27 03:55:39,789 INFO [StoreOpener-95b223459399eec298605c6ce5cd65f6-1] regionserver.HStore(310): Store=95b223459399eec298605c6ce5cd65f6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:55:39,789 INFO [StoreOpener-95b223459399eec298605c6ce5cd65f6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 95b223459399eec298605c6ce5cd65f6 2023-03-27 03:55:39,790 DEBUG [StoreOpener-95b223459399eec298605c6ce5cd65f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2 2023-03-27 03:55:39,790 DEBUG [StoreOpener-95b223459399eec298605c6ce5cd65f6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2 2023-03-27 03:55:39,790 INFO [StoreOpener-95b223459399eec298605c6ce5cd65f6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 95b223459399eec298605c6ce5cd65f6 columnFamilyName family2 2023-03-27 03:55:39,791 INFO [StoreOpener-95b223459399eec298605c6ce5cd65f6-1] regionserver.HStore(310): Store=95b223459399eec298605c6ce5cd65f6/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:55:39,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6 2023-03-27 03:55:39,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6 2023-03-27 03:55:39,794 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-27 03:55:39,795 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 95b223459399eec298605c6ce5cd65f6 2023-03-27 03:55:39,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:55:39,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 95b223459399eec298605c6ce5cd65f6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74495617, jitterRate=0.11007119715213776}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-27 03:55:39,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 95b223459399eec298605c6ce5cd65f6: 2023-03-27 03:55:39,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6., pid=120, masterSystemTime=1679889339711 2023-03-27 03:55:39,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:39,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:39,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:39,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d0bf0165da69488ffc28ab9679dbd8e9, NAME => 'testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9.', STARTKEY => '2', ENDKEY => ''} 2023-03-27 03:55:39,799 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=95b223459399eec298605c6ce5cd65f6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:39,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin d0bf0165da69488ffc28ab9679dbd8e9 2023-03-27 03:55:39,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-03-27 03:55:39,799 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679889339799"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889339799"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889339799"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889339799"}]},"ts":"1679889339799"} 2023-03-27 03:55:39,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d0bf0165da69488ffc28ab9679dbd8e9 2023-03-27 03:55:39,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d0bf0165da69488ffc28ab9679dbd8e9 2023-03-27 03:55:39,802 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=117 2023-03-27 03:55:39,802 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=117, state=SUCCESS; OpenRegionProcedure 95b223459399eec298605c6ce5cd65f6, server=jenkins-hbase4.apache.org,39727,1679889218560 in 241 msec 2023-03-27 03:55:39,803 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=95b223459399eec298605c6ce5cd65f6, ASSIGN in 399 msec 2023-03-27 03:55:39,823 INFO [StoreOpener-d0bf0165da69488ffc28ab9679dbd8e9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d0bf0165da69488ffc28ab9679dbd8e9 2023-03-27 03:55:39,824 DEBUG [StoreOpener-d0bf0165da69488ffc28ab9679dbd8e9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1 2023-03-27 03:55:39,825 DEBUG [StoreOpener-d0bf0165da69488ffc28ab9679dbd8e9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1 2023-03-27 03:55:39,825 INFO [StoreOpener-d0bf0165da69488ffc28ab9679dbd8e9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d0bf0165da69488ffc28ab9679dbd8e9 columnFamilyName family1 2023-03-27 03:55:39,825 INFO [StoreOpener-d0bf0165da69488ffc28ab9679dbd8e9-1] regionserver.HStore(310): Store=d0bf0165da69488ffc28ab9679dbd8e9/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:55:39,826 INFO [StoreOpener-d0bf0165da69488ffc28ab9679dbd8e9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region d0bf0165da69488ffc28ab9679dbd8e9 2023-03-27 03:55:39,827 DEBUG [StoreOpener-d0bf0165da69488ffc28ab9679dbd8e9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2 2023-03-27 03:55:39,827 DEBUG [StoreOpener-d0bf0165da69488ffc28ab9679dbd8e9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2 2023-03-27 03:55:39,827 INFO [StoreOpener-d0bf0165da69488ffc28ab9679dbd8e9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d0bf0165da69488ffc28ab9679dbd8e9 columnFamilyName family2 2023-03-27 03:55:39,828 INFO [StoreOpener-d0bf0165da69488ffc28ab9679dbd8e9-1] regionserver.HStore(310): Store=d0bf0165da69488ffc28ab9679dbd8e9/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-03-27 03:55:39,828 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9 2023-03-27 03:55:39,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9 2023-03-27 03:55:39,830 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-03-27 03:55:39,830 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d0bf0165da69488ffc28ab9679dbd8e9 2023-03-27 03:55:39,832 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-03-27 03:55:39,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d0bf0165da69488ffc28ab9679dbd8e9; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66295639, jitterRate=-0.01211799681186676}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-03-27 03:55:39,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d0bf0165da69488ffc28ab9679dbd8e9: 2023-03-27 03:55:39,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9., pid=122, masterSystemTime=1679889339711 2023-03-27 03:55:39,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:39,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:39,835 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=d0bf0165da69488ffc28ab9679dbd8e9, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:39,835 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1679889339835"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1679889339835"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1679889339835"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1679889339835"}]},"ts":"1679889339835"} 2023-03-27 03:55:39,837 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=119 2023-03-27 03:55:39,837 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=119, state=SUCCESS; OpenRegionProcedure d0bf0165da69488ffc28ab9679dbd8e9, server=jenkins-hbase4.apache.org,39727,1679889218560 in 276 msec 2023-03-27 03:55:39,839 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=119, resume processing ppid=116 2023-03-27 03:55:39,839 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=d0bf0165da69488ffc28ab9679dbd8e9, ASSIGN in 434 msec 2023-03-27 03:55:39,839 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-03-27 03:55:39,839 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1679889339839"}]},"ts":"1679889339839"} 2023-03-27 03:55:39,840 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-03-27 03:55:39,843 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-03-27 03:55:39,844 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 1.2680 sec 2023-03-27 03:55:40,289 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-03-27 03:55:40,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-03-27 03:55:40,682 INFO [Listener at localhost/36131] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-03-27 03:55:40,689 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:40,689 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:40,690 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:40,692 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:40,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-27 03:55:40,695 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-27 03:55:40,695 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:40,695 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-27 03:55:40,695 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-27 03:55:40,695 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:40,695 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:40,704 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:40,704 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:40,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:40,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:40,705 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:40,705 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:40,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:40,705 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:40,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:40,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:40,705 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-27 03:55:40,706 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-27 03:55:40,706 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-27 03:55:40,706 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:40,706 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-27 03:55:40,706 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:40,706 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. started... 2023-03-27 03:55:40,706 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:40,706 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. started... 2023-03-27 03:55:40,706 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-27 03:55:40,707 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:40,708 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:40,708 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. started... 2023-03-27 03:55:40,708 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ae7919e6ac686a9f3a2be09b01a0ff1a 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-27 03:55:40,709 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 95b223459399eec298605c6ce5cd65f6 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-27 03:55:40,711 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d0bf0165da69488ffc28ab9679dbd8e9 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-03-27 03:55:40,806 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230327eba97634e826417b9cc6aa8522c4bc0f to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230327eba97634e826417b9cc6aa8522c4bc0f 2023-03-27 03:55:40,809 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032798398f24982c45308ef8827e1e50ac21 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032798398f24982c45308ef8827e1e50ac21 2023-03-27 03:55:40,840 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family1/1258ec316a744477bf94485bf4ff163d 2023-03-27 03:55:40,844 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family1/b2a7d03d478c415591b3af0413c70978 2023-03-27 03:55:40,898 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230327cf12abc196764a708bdac2da105448bf to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230327cf12abc196764a708bdac2da105448bf 2023-03-27 03:55:41,209 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230327fa5cc888337540ef8088eaf94ce0023f to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230327fa5cc888337540ef8088eaf94ce0023f 2023-03-27 03:55:41,230 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family1/690f82499a7d4d4cb35fa108a740f436 2023-03-27 03:55:41,299 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230327f1e024fc7043455aa00bf22b706f41a6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230327f1e024fc7043455aa00bf22b706f41a6 2023-03-27 03:55:41,304 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family2/8289773ba559468fadc4e4bd8d7251eb 2023-03-27 03:55:41,308 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family1/1258ec316a744477bf94485bf4ff163d as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/1258ec316a744477bf94485bf4ff163d 2023-03-27 03:55:41,332 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/1258ec316a744477bf94485bf4ff163d, entries=4, sequenceid=5, filesize=5.5 K 2023-03-27 03:55:41,333 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family2/8289773ba559468fadc4e4bd8d7251eb as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/8289773ba559468fadc4e4bd8d7251eb 2023-03-27 03:55:41,336 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/8289773ba559468fadc4e4bd8d7251eb, entries=2, sequenceid=5, filesize=5.1 K 2023-03-27 03:55:41,337 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for d0bf0165da69488ffc28ab9679dbd8e9 in 627ms, sequenceid=5, compaction requested=false 2023-03-27 03:55:41,337 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-03-27 03:55:41,337 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d0bf0165da69488ffc28ab9679dbd8e9: 2023-03-27 03:55:41,337 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:41,337 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-27 03:55:41,665 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230327d9d98854206d44ed9f95ab9ccacc25bf to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230327d9d98854206d44ed9f95ab9ccacc25bf 2023-03-27 03:55:41,704 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family2/0d554fb49a0c4852ab82d8a8b0fe2449 2023-03-27 03:55:41,710 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family1/b2a7d03d478c415591b3af0413c70978 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/b2a7d03d478c415591b3af0413c70978 2023-03-27 03:55:41,740 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/b2a7d03d478c415591b3af0413c70978, entries=4, sequenceid=5, filesize=5.5 K 2023-03-27 03:55:41,741 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family2/0d554fb49a0c4852ab82d8a8b0fe2449 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/0d554fb49a0c4852ab82d8a8b0fe2449 2023-03-27 03:55:41,746 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/0d554fb49a0c4852ab82d8a8b0fe2449, entries=2, sequenceid=5, filesize=5.1 K 2023-03-27 03:55:41,747 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for ae7919e6ac686a9f3a2be09b01a0ff1a in 1039ms, sequenceid=5, compaction requested=false 2023-03-27 03:55:41,748 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ae7919e6ac686a9f3a2be09b01a0ff1a: 2023-03-27 03:55:41,748 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:41,748 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-27 03:55:42,072 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family2/af6fe675c27f417d84fd72b8dd702508 2023-03-27 03:55:42,075 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family1/690f82499a7d4d4cb35fa108a740f436 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/690f82499a7d4d4cb35fa108a740f436 2023-03-27 03:55:42,101 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/690f82499a7d4d4cb35fa108a740f436, entries=4, sequenceid=5, filesize=5.5 K 2023-03-27 03:55:42,102 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family2/af6fe675c27f417d84fd72b8dd702508 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/af6fe675c27f417d84fd72b8dd702508 2023-03-27 03:55:42,105 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/af6fe675c27f417d84fd72b8dd702508, entries=2, sequenceid=5, filesize=5.1 K 2023-03-27 03:55:42,106 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 95b223459399eec298605c6ce5cd65f6 in 1397ms, sequenceid=5, compaction requested=false 2023-03-27 03:55:42,106 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 95b223459399eec298605c6ce5cd65f6: 2023-03-27 03:55:42,106 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:42,106 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-27 03:55:42,107 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-27 03:55:42,107 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:42,107 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-27 03:55:42,107 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-27 03:55:42,108 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:42,108 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:42,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:42,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:42,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:42,109 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:42,109 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:42,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:42,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:42,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:42,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:42,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:42,110 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-27 03:55:42,110 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@22280a9f[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:42,110 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-27 03:55:42,110 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:42,112 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:42,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:42,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:42,113 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-27 03:55:42,113 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-27 03:55:42,113 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:42,113 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:42,113 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:42,114 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:42,114 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-27 03:55:42,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:42,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:42,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:42,114 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:42,114 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-27 03:55:42,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:42,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:42,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:42,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:42,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:42,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:42,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:42,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:42,116 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-27 03:55:42,116 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-27 03:55:42,116 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:42,116 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:42,116 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-27 03:55:42,116 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:42,118 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:42,118 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:42,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:42,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:42,118 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:42,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:42,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:42,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:42,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:42,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:42,118 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:42,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:42,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:42,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:42,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:42,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:42,119 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:42,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:42,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:42,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:42,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:42,121 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:42,121 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:42,121 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:42,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:42,121 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:42,121 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:42,121 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:42,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:42,121 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:42,121 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:42,121 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-27 03:55:42,122 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:42,122 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:42,122 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:42,122 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-27 03:55:42,122 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:42,122 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:42,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:42,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:42,222 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:42,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:42,224 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:42,224 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:42,224 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:42,226 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:42,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-27 03:55:42,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-27 03:55:42,228 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-27 03:55:42,228 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:42,228 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-27 03:55:42,228 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-27 03:55:42,228 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:42,228 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:42,230 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:42,230 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:42,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:42,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:42,230 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:42,230 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:42,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:42,231 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:42,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:42,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:42,231 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-27 03:55:42,231 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-27 03:55:42,231 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-27 03:55:42,232 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-27 03:55:42,232 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:42,232 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-27 03:55:42,232 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:42,232 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-27 03:55:42,232 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:42,232 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. started... 2023-03-27 03:55:42,232 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:42,232 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:42,232 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 95b223459399eec298605c6ce5cd65f6 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-27 03:55:42,232 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. started... 2023-03-27 03:55:42,232 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. started... 2023-03-27 03:55:42,233 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ae7919e6ac686a9f3a2be09b01a0ff1a 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-27 03:55:42,233 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d0bf0165da69488ffc28ab9679dbd8e9 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-03-27 03:55:42,341 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230327b721cdc2ad734e98a633e281bf0f263f to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230327b721cdc2ad734e98a633e281bf0f263f 2023-03-27 03:55:42,343 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230327174b3b347db6493db21425cfa5754d9c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230327174b3b347db6493db21425cfa5754d9c 2023-03-27 03:55:42,386 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230327d315634bec454bca99b62789046d98ec to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230327d315634bec454bca99b62789046d98ec 2023-03-27 03:55:42,408 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family1/e1e70a6f124a440ba818da5914b56d51 2023-03-27 03:55:42,430 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family1/61e706d4da9642848d106bb629cecf86 2023-03-27 03:55:42,462 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230327ae1188118c8742ce9da7a449f0395dbe to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230327ae1188118c8742ce9da7a449f0395dbe 2023-03-27 03:55:42,492 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032787f6322d96de4bbfb36a196998dd0022 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023032787f6322d96de4bbfb36a196998dd0022 2023-03-27 03:55:42,496 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family2/35b9c7792772403cac8a65bbc4aa43fd 2023-03-27 03:55:42,501 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family1/61e706d4da9642848d106bb629cecf86 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/61e706d4da9642848d106bb629cecf86 2023-03-27 03:55:42,532 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/61e706d4da9642848d106bb629cecf86, entries=4, sequenceid=9, filesize=5.5 K 2023-03-27 03:55:42,533 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family2/35b9c7792772403cac8a65bbc4aa43fd as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/35b9c7792772403cac8a65bbc4aa43fd 2023-03-27 03:55:42,537 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/35b9c7792772403cac8a65bbc4aa43fd, entries=2, sequenceid=9, filesize=5.1 K 2023-03-27 03:55:42,537 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 95b223459399eec298605c6ce5cd65f6 in 305ms, sequenceid=9, compaction requested=false 2023-03-27 03:55:42,538 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 95b223459399eec298605c6ce5cd65f6: 2023-03-27 03:55:42,538 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:42,538 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-27 03:55:42,789 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family1/534e0b6bdbfe47c59e6f128d45748c06 2023-03-27 03:55:42,829 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230327f9ad51e82ca5481ea6ba6323ae4b6b9d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230327f9ad51e82ca5481ea6ba6323ae4b6b9d 2023-03-27 03:55:42,875 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family2/d68dc07e609d49b2820d6c42367cc852 2023-03-27 03:55:42,880 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family1/e1e70a6f124a440ba818da5914b56d51 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/e1e70a6f124a440ba818da5914b56d51 2023-03-27 03:55:42,917 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/e1e70a6f124a440ba818da5914b56d51, entries=4, sequenceid=9, filesize=5.5 K 2023-03-27 03:55:42,918 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family2/d68dc07e609d49b2820d6c42367cc852 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/d68dc07e609d49b2820d6c42367cc852 2023-03-27 03:55:42,923 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/d68dc07e609d49b2820d6c42367cc852, entries=2, sequenceid=9, filesize=5.1 K 2023-03-27 03:55:42,924 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for ae7919e6ac686a9f3a2be09b01a0ff1a in 691ms, sequenceid=9, compaction requested=false 2023-03-27 03:55:42,924 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ae7919e6ac686a9f3a2be09b01a0ff1a: 2023-03-27 03:55:42,924 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:42,924 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-27 03:55:43,233 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family2/18ecda24e0de4a0e93ba368697161804 2023-03-27 03:55:43,238 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family1/534e0b6bdbfe47c59e6f128d45748c06 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/534e0b6bdbfe47c59e6f128d45748c06 2023-03-27 03:55:43,264 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/534e0b6bdbfe47c59e6f128d45748c06, entries=4, sequenceid=9, filesize=5.5 K 2023-03-27 03:55:43,265 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family2/18ecda24e0de4a0e93ba368697161804 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/18ecda24e0de4a0e93ba368697161804 2023-03-27 03:55:43,269 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/18ecda24e0de4a0e93ba368697161804, entries=2, sequenceid=9, filesize=5.1 K 2023-03-27 03:55:43,270 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for d0bf0165da69488ffc28ab9679dbd8e9 in 1037ms, sequenceid=9, compaction requested=false 2023-03-27 03:55:43,270 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d0bf0165da69488ffc28ab9679dbd8e9: 2023-03-27 03:55:43,270 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:43,270 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-27 03:55:43,270 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-27 03:55:43,270 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:43,270 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-27 03:55:43,271 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-27 03:55:43,273 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:43,273 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:43,273 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:43,274 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:43,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:43,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:43,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:43,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:43,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:43,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:43,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:43,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:43,276 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-27 03:55:43,276 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@36fd562a[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:43,276 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-27 03:55:43,277 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:43,278 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:43,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:43,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:43,278 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-27 03:55:43,278 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-27 03:55:43,278 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:43,278 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:43,278 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:43,279 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:43,279 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-27 03:55:43,279 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:43,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:43,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:43,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:43,279 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-27 03:55:43,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:43,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:43,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:43,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:43,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:43,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:43,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:43,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:43,283 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-27 03:55:43,283 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-27 03:55:43,283 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:43,283 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:43,283 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-27 03:55:43,283 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:43,285 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:43,285 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:43,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:43,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:43,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:43,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:43,286 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:43,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:43,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:43,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:43,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:43,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:43,286 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:43,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:43,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:43,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:43,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:43,288 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:43,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:43,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:43,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:43,290 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:43,290 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:43,290 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:43,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:43,290 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:43,290 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-27 03:55:43,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:43,290 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:43,290 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:43,291 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:43,291 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:43,291 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:43,291 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:43,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:43,291 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:43,291 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:43,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:43,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:43,391 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:43,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:43,393 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:43,393 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:43,394 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:43,396 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:43,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-27 03:55:43,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-27 03:55:43,398 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-27 03:55:43,399 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:43,399 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-27 03:55:43,399 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-27 03:55:43,399 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:43,399 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:43,401 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:43,401 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:43,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:43,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:43,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:43,401 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:43,401 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:43,402 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:43,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:43,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:43,402 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-27 03:55:43,402 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-27 03:55:43,402 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-27 03:55:43,402 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-27 03:55:43,402 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:43,402 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-27 03:55:43,403 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:43,403 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. started... 2023-03-27 03:55:43,403 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 95b223459399eec298605c6ce5cd65f6 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-27 03:55:43,408 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:43,408 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-27 03:55:43,408 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:43,408 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:43,408 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. started... 2023-03-27 03:55:43,408 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. started... 2023-03-27 03:55:43,408 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ae7919e6ac686a9f3a2be09b01a0ff1a 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-27 03:55:43,408 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d0bf0165da69488ffc28ab9679dbd8e9 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-03-27 03:55:43,510 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303273593cf738f004068a64c899f122fe939 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303273593cf738f004068a64c899f122fe939 2023-03-27 03:55:43,513 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032777f94cb8477b4c24a0cc2b6370a8be5b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032777f94cb8477b4c24a0cc2b6370a8be5b 2023-03-27 03:55:43,895 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023032745c61f79664b4b7fb6674a9df6d9ca7d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023032745c61f79664b4b7fb6674a9df6d9ca7d 2023-03-27 03:55:43,948 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family1/5deb15061f514a1c83724578fddecfcd 2023-03-27 03:55:43,954 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family1/889fd089a0e54b0aae35e55caeb9790b 2023-03-27 03:55:44,011 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202303274937d29c563647208c765eb4411ec03a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202303274937d29c563647208c765eb4411ec03a 2023-03-27 03:55:44,017 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family2/a1c2a153dcbc4b67a6b61e83dfd4d27b 2023-03-27 03:55:44,020 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family1/889fd089a0e54b0aae35e55caeb9790b as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/889fd089a0e54b0aae35e55caeb9790b 2023-03-27 03:55:44,048 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/889fd089a0e54b0aae35e55caeb9790b, entries=4, sequenceid=13, filesize=5.5 K 2023-03-27 03:55:44,048 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family2/a1c2a153dcbc4b67a6b61e83dfd4d27b as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/a1c2a153dcbc4b67a6b61e83dfd4d27b 2023-03-27 03:55:44,052 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/a1c2a153dcbc4b67a6b61e83dfd4d27b, entries=2, sequenceid=13, filesize=5.1 K 2023-03-27 03:55:44,054 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for d0bf0165da69488ffc28ab9679dbd8e9 in 646ms, sequenceid=13, compaction requested=true 2023-03-27 03:55:44,054 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d0bf0165da69488ffc28ab9679dbd8e9: 2023-03-27 03:55:44,054 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:44,054 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-27 03:55:44,316 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family1/c2350ce460694fa180dafc621be33c0a 2023-03-27 03:55:44,418 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230327c004813fd5534ab587a84c995d8e8a04 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230327c004813fd5534ab587a84c995d8e8a04 2023-03-27 03:55:44,759 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230327246cd862fd144b67ab23d29d3f00a2fe to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230327246cd862fd144b67ab23d29d3f00a2fe 2023-03-27 03:55:44,823 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family2/6571508ba8654b42877a9f298086d0de 2023-03-27 03:55:44,827 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family1/5deb15061f514a1c83724578fddecfcd as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/5deb15061f514a1c83724578fddecfcd 2023-03-27 03:55:44,853 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/5deb15061f514a1c83724578fddecfcd, entries=4, sequenceid=13, filesize=5.5 K 2023-03-27 03:55:44,854 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family2/6571508ba8654b42877a9f298086d0de as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/6571508ba8654b42877a9f298086d0de 2023-03-27 03:55:44,863 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/6571508ba8654b42877a9f298086d0de, entries=2, sequenceid=13, filesize=5.1 K 2023-03-27 03:55:44,863 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for ae7919e6ac686a9f3a2be09b01a0ff1a in 1455ms, sequenceid=13, compaction requested=true 2023-03-27 03:55:44,864 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ae7919e6ac686a9f3a2be09b01a0ff1a: 2023-03-27 03:55:44,864 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:44,864 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-27 03:55:45,165 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family2/5513b324d73943e9842e2085f048d9cb 2023-03-27 03:55:45,170 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family1/c2350ce460694fa180dafc621be33c0a as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/c2350ce460694fa180dafc621be33c0a 2023-03-27 03:55:45,203 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/c2350ce460694fa180dafc621be33c0a, entries=4, sequenceid=13, filesize=5.5 K 2023-03-27 03:55:45,204 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family2/5513b324d73943e9842e2085f048d9cb as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/5513b324d73943e9842e2085f048d9cb 2023-03-27 03:55:45,208 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/5513b324d73943e9842e2085f048d9cb, entries=2, sequenceid=13, filesize=5.1 K 2023-03-27 03:55:45,209 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 95b223459399eec298605c6ce5cd65f6 in 1806ms, sequenceid=13, compaction requested=true 2023-03-27 03:55:45,209 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 95b223459399eec298605c6ce5cd65f6: 2023-03-27 03:55:45,209 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:45,209 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-27 03:55:45,209 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-27 03:55:45,209 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:45,209 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-27 03:55:45,209 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-27 03:55:45,211 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:45,211 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:45,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:45,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:45,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:45,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:45,212 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:45,212 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:45,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:45,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:45,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:45,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:45,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-27 03:55:45,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@392bd9bf[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:45,213 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-27 03:55:45,213 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:45,214 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:45,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:45,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:45,215 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-27 03:55:45,215 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-27 03:55:45,215 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:45,216 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-03-27 03:55:45,217 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:45,217 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-27 03:55:45,217 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:45,217 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-27 03:55:45,217 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:45,217 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:45,217 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-27 03:55:45,217 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:45,218 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-27 03:55:45,218 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:45,218 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-27 03:55:45,219 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:45,219 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:45,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:45,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:45,219 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:45,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:45,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:45,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:45,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:45,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:45,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:45,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:45,220 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:45,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:45,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:45,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:45,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:45,222 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:45,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:45,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:45,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:45,232 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:45,232 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:45,232 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:45,232 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:45,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:45,232 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-27 03:55:45,232 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:45,232 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:45,232 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:45,232 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:45,232 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:45,232 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:45,232 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:45,232 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:45,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:45,233 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:45,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:45,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:45,286 INFO [Finalizer] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-27 03:55:45,286 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x425b86ea to 127.0.0.1:62246 2023-03-27 03:55:45,286 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:55:45,333 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:45,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:45,334 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:45,335 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:45,335 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. with WAL disabled. Data may be lost in the event of a crash. 2023-03-27 03:55:45,337 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:45,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-27 03:55:45,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-27 03:55:45,339 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-27 03:55:45,339 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:45,339 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-27 03:55:45,339 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-27 03:55:45,339 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:45,339 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:45,344 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:45,344 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:45,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:45,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:45,344 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:45,344 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:45,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:45,345 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:45,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:45,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:45,345 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-27 03:55:45,345 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-27 03:55:45,345 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-27 03:55:45,345 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:45,345 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-27 03:55:45,345 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:45,345 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. started... 2023-03-27 03:55:45,346 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:45,346 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 95b223459399eec298605c6ce5cd65f6 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-27 03:55:45,346 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. started... 2023-03-27 03:55:45,346 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-27 03:55:45,346 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:45,346 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ae7919e6ac686a9f3a2be09b01a0ff1a 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-27 03:55:45,346 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:45,346 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. started... 2023-03-27 03:55:45,346 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d0bf0165da69488ffc28ab9679dbd8e9 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-03-27 03:55:45,409 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303277c93768bbd0e4f3a972eb9c770bf9b45 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303277c93768bbd0e4f3a972eb9c770bf9b45 2023-03-27 03:55:45,425 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family1/bdc559262daf42b7bca182ae7ef2a2b5 2023-03-27 03:55:45,463 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303270d5bee0a82044bddaa3c5cfddc3aefd6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202303270d5bee0a82044bddaa3c5cfddc3aefd6 2023-03-27 03:55:45,469 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family2/423ccbcd803544e0840e460a2130b169 2023-03-27 03:55:45,473 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family1/bdc559262daf42b7bca182ae7ef2a2b5 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/bdc559262daf42b7bca182ae7ef2a2b5 2023-03-27 03:55:45,513 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/bdc559262daf42b7bca182ae7ef2a2b5, entries=4, sequenceid=17, filesize=5.5 K 2023-03-27 03:55:45,514 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family2/423ccbcd803544e0840e460a2130b169 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/423ccbcd803544e0840e460a2130b169 2023-03-27 03:55:45,520 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/423ccbcd803544e0840e460a2130b169, entries=2, sequenceid=17, filesize=5.1 K 2023-03-27 03:55:45,521 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for ae7919e6ac686a9f3a2be09b01a0ff1a in 175ms, sequenceid=17, compaction requested=true 2023-03-27 03:55:45,521 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ae7919e6ac686a9f3a2be09b01a0ff1a: 2023-03-27 03:55:45,521 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:45,521 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-27 03:55:45,813 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230327c6dabeb9f1bb442c95bb45971deadead to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230327c6dabeb9f1bb442c95bb45971deadead 2023-03-27 03:55:45,820 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230327b29087d566ef4b7bae7cc7e09d26c8ba to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230327b29087d566ef4b7bae7cc7e09d26c8ba 2023-03-27 03:55:45,844 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family1/090c02539bea4b2885550efe2dc12cce 2023-03-27 03:55:45,853 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family1/fb5bc9014897411faf086ce8adbe7351 2023-03-27 03:55:45,911 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230327eeb8359a5de24c789f432d660d1c3ee2 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230327eeb8359a5de24c789f432d660d1c3ee2 2023-03-27 03:55:46,317 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family2/982d8a5c6b544a91a61cfc38abfaecdb 2023-03-27 03:55:46,318 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230327fffd64e2468d4257a4bcb80a744d3ab1 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230327fffd64e2468d4257a4bcb80a744d3ab1 2023-03-27 03:55:46,322 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family1/090c02539bea4b2885550efe2dc12cce as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/090c02539bea4b2885550efe2dc12cce 2023-03-27 03:55:46,361 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/090c02539bea4b2885550efe2dc12cce, entries=4, sequenceid=17, filesize=5.5 K 2023-03-27 03:55:46,364 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family2/982d8a5c6b544a91a61cfc38abfaecdb as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/982d8a5c6b544a91a61cfc38abfaecdb 2023-03-27 03:55:46,368 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/982d8a5c6b544a91a61cfc38abfaecdb, entries=2, sequenceid=17, filesize=5.1 K 2023-03-27 03:55:46,369 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for d0bf0165da69488ffc28ab9679dbd8e9 in 1023ms, sequenceid=17, compaction requested=true 2023-03-27 03:55:46,369 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d0bf0165da69488ffc28ab9679dbd8e9: 2023-03-27 03:55:46,369 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:46,369 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-27 03:55:46,723 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family2/a1eb2586c1c24ca6bb042a425dc913e6 2023-03-27 03:55:46,727 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family1/fb5bc9014897411faf086ce8adbe7351 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/fb5bc9014897411faf086ce8adbe7351 2023-03-27 03:55:46,755 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/fb5bc9014897411faf086ce8adbe7351, entries=4, sequenceid=17, filesize=5.5 K 2023-03-27 03:55:46,756 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family2/a1eb2586c1c24ca6bb042a425dc913e6 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/a1eb2586c1c24ca6bb042a425dc913e6 2023-03-27 03:55:46,760 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/a1eb2586c1c24ca6bb042a425dc913e6, entries=2, sequenceid=17, filesize=5.1 K 2023-03-27 03:55:46,761 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 95b223459399eec298605c6ce5cd65f6 in 1415ms, sequenceid=17, compaction requested=true 2023-03-27 03:55:46,761 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 95b223459399eec298605c6ce5cd65f6: 2023-03-27 03:55:46,761 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:46,761 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-27 03:55:46,761 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-27 03:55:46,761 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:46,761 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-27 03:55:46,761 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-27 03:55:46,763 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:46,763 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:46,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:46,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:46,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:46,764 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:46,764 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:46,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:46,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:46,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:46,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:46,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:46,766 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-27 03:55:46,766 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ac01f9f[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:46,766 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-27 03:55:46,766 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:46,767 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:46,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:46,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:46,767 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-27 03:55:46,767 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-27 03:55:46,767 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:46,767 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:46,767 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:46,769 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:46,769 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-27 03:55:46,769 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:46,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:46,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:46,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:46,769 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-27 03:55:46,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:46,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:46,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:46,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:46,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:46,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:46,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:46,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:46,771 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-27 03:55:46,771 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-27 03:55:46,771 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:46,771 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:46,772 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-27 03:55:46,772 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:46,773 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:46,773 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:46,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:46,773 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:46,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:46,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:46,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:46,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:46,774 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:46,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:46,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:46,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:46,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:46,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:46,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:46,775 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:46,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:46,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:46,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:46,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:46,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:46,778 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:46,778 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:46,779 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:46,778 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:46,779 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:46,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:46,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:46,779 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:46,779 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:46,779 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:46,779 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:46,779 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-27 03:55:46,779 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:46,779 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:46,779 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-27 03:55:46,780 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:46,780 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:46,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:46,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:46,880 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:46,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:46,890 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-27 03:55:46,898 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-27 03:55:46,907 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-27 03:55:46,908 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-27 03:55:47,030 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-03-27 03:55:47,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-03-27 03:55:47,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-03-27 03:55:47,032 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-03-27 03:55:47,032 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:47,032 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-03-27 03:55:47,032 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-03-27 03:55:47,032 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:47,032 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:47,034 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:47,034 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:47,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:47,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:47,034 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:47,034 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-03-27 03:55:47,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:47,036 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:47,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-03-27 03:55:47,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:47,036 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-03-27 03:55:47,036 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-03-27 03:55:47,037 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-03-27 03:55:47,037 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-03-27 03:55:47,037 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-03-27 03:55:47,037 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-03-27 03:55:47,037 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:47,037 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. started... 2023-03-27 03:55:47,037 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 95b223459399eec298605c6ce5cd65f6 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-27 03:55:47,038 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:47,038 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. started... 2023-03-27 03:55:47,038 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ae7919e6ac686a9f3a2be09b01a0ff1a 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-27 03:55:47,038 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-03-27 03:55:47,039 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-03-27 03:55:47,039 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:47,039 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. started... 2023-03-27 03:55:47,039 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing d0bf0165da69488ffc28ab9679dbd8e9 2/2 column families, dataSize=138 B heapSize=944 B 2023-03-27 03:55:47,118 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family1/ebbb196f8d8945b9aa4b100a7c5a57ea 2023-03-27 03:55:47,145 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ebbb196f8d8945b9aa4b100a7c5a57ea 2023-03-27 03:55:47,497 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family1/84eaf212dc844879bc2f8c7bdab0962b 2023-03-27 03:55:47,519 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family1/21bfc35c25b84019b6fd753b46e8be23 2023-03-27 03:55:47,526 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 84eaf212dc844879bc2f8c7bdab0962b 2023-03-27 03:55:47,549 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 21bfc35c25b84019b6fd753b46e8be23 2023-03-27 03:55:47,568 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family2/7f470426775349f5bbe2a5b10ce13af3 2023-03-27 03:55:47,572 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7f470426775349f5bbe2a5b10ce13af3 2023-03-27 03:55:47,573 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family1/ebbb196f8d8945b9aa4b100a7c5a57ea as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/ebbb196f8d8945b9aa4b100a7c5a57ea 2023-03-27 03:55:47,607 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ebbb196f8d8945b9aa4b100a7c5a57ea 2023-03-27 03:55:47,607 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/ebbb196f8d8945b9aa4b100a7c5a57ea, entries=3, sequenceid=23, filesize=5.2 K 2023-03-27 03:55:47,608 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family2/7f470426775349f5bbe2a5b10ce13af3 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/7f470426775349f5bbe2a5b10ce13af3 2023-03-27 03:55:47,613 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7f470426775349f5bbe2a5b10ce13af3 2023-03-27 03:55:47,613 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/7f470426775349f5bbe2a5b10ce13af3, entries=1, sequenceid=23, filesize=5.0 K 2023-03-27 03:55:47,614 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for d0bf0165da69488ffc28ab9679dbd8e9 in 575ms, sequenceid=23, compaction requested=true 2023-03-27 03:55:47,614 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for d0bf0165da69488ffc28ab9679dbd8e9: 2023-03-27 03:55:47,614 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:47,614 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-03-27 03:55:47,958 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family2/7a1ae30890db498fa2fb7723b9df6fd7 2023-03-27 03:55:47,961 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7a1ae30890db498fa2fb7723b9df6fd7 2023-03-27 03:55:47,962 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family1/84eaf212dc844879bc2f8c7bdab0962b as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/84eaf212dc844879bc2f8c7bdab0962b 2023-03-27 03:55:47,978 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family2/2ca56f349799475abc628515e864919c 2023-03-27 03:55:47,982 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ca56f349799475abc628515e864919c 2023-03-27 03:55:47,982 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family1/21bfc35c25b84019b6fd753b46e8be23 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/21bfc35c25b84019b6fd753b46e8be23 2023-03-27 03:55:47,986 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 84eaf212dc844879bc2f8c7bdab0962b 2023-03-27 03:55:47,986 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/84eaf212dc844879bc2f8c7bdab0962b, entries=3, sequenceid=23, filesize=5.2 K 2023-03-27 03:55:47,986 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family2/7a1ae30890db498fa2fb7723b9df6fd7 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/7a1ae30890db498fa2fb7723b9df6fd7 2023-03-27 03:55:47,990 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7a1ae30890db498fa2fb7723b9df6fd7 2023-03-27 03:55:47,990 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/7a1ae30890db498fa2fb7723b9df6fd7, entries=1, sequenceid=23, filesize=5.0 K 2023-03-27 03:55:47,991 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for ae7919e6ac686a9f3a2be09b01a0ff1a in 953ms, sequenceid=23, compaction requested=true 2023-03-27 03:55:47,991 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ae7919e6ac686a9f3a2be09b01a0ff1a: 2023-03-27 03:55:47,991 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:47,991 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-03-27 03:55:48,007 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 21bfc35c25b84019b6fd753b46e8be23 2023-03-27 03:55:48,007 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/21bfc35c25b84019b6fd753b46e8be23, entries=3, sequenceid=23, filesize=5.2 K 2023-03-27 03:55:48,008 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family2/2ca56f349799475abc628515e864919c as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/2ca56f349799475abc628515e864919c 2023-03-27 03:55:48,012 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ca56f349799475abc628515e864919c 2023-03-27 03:55:48,012 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/2ca56f349799475abc628515e864919c, entries=1, sequenceid=23, filesize=5.0 K 2023-03-27 03:55:48,013 INFO [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 95b223459399eec298605c6ce5cd65f6 in 976ms, sequenceid=23, compaction requested=true 2023-03-27 03:55:48,015 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 95b223459399eec298605c6ce5cd65f6: 2023-03-27 03:55:48,015 DEBUG [rs(jenkins-hbase4.apache.org,39727,1679889218560)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:48,015 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-03-27 03:55:48,015 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-03-27 03:55:48,015 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:48,015 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-03-27 03:55:48,015 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-03-27 03:55:48,017 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:48,017 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:48,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:48,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:48,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:48,017 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:48,018 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-03-27 03:55:48,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:48,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:48,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:48,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:48,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:48,020 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,39727,1679889218560' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-03-27 03:55:48,020 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38de9aaa[Count = 0] remaining members to acquire global barrier 2023-03-27 03:55:48,020 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-03-27 03:55:48,020 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:48,021 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:48,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:48,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:48,021 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-03-27 03:55:48,021 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-03-27 03:55:48,021 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,39727,1679889218560' in zk 2023-03-27 03:55:48,021 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:48,021 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-03-27 03:55:48,024 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:48,024 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-03-27 03:55:48,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:48,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:48,024 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:48,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:48,024 DEBUG [member: 'jenkins-hbase4.apache.org,39727,1679889218560' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-03-27 03:55:48,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:48,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:48,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:48,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:48,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:48,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:48,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:48,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,39727,1679889218560': 2023-03-27 03:55:48,026 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,39727,1679889218560' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-03-27 03:55:48,026 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-03-27 03:55:48,026 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-03-27 03:55:48,026 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-03-27 03:55:48,026 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-03-27 03:55:48,026 INFO [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-03-27 03:55:48,028 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:48,028 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:48,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:48,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:48,028 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:48,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:48,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-03-27 03:55:48,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-03-27 03:55:48,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:48,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:48,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-03-27 03:55:48,029 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:48,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:48,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:48,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-03-27 03:55:48,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:48,030 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:48,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:48,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-03-27 03:55:48,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-03-27 03:55:48,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:48,032 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:48,032 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:48,032 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-03-27 03:55:48,032 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:48,032 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:48,032 DEBUG [(jenkins-hbase4.apache.org,38271,1679889217353)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-03-27 03:55:48,032 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-03-27 03:55:48,032 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-03-27 03:55:48,032 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-03-27 03:55:48,032 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-03-27 03:55:48,032 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-03-27 03:55:48,033 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-03-27 03:55:48,033 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-03-27 03:55:48,033 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-03-27 03:55:48,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-03-27 03:55:48,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-03-27 03:55:48,033 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-03-27 03:55:48,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-03-27 03:55:48,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-03-27 03:55:48,133 DEBUG [Listener at localhost/36131] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-03-27 03:55:48,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38271] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-03-27 03:55:48,134 DEBUG [Listener at localhost/36131] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-27 03:55:48,134 DEBUG [Listener at localhost/36131] regionserver.HStore(1912): 95b223459399eec298605c6ce5cd65f6/family1 is initiating major compaction (all files) 2023-03-27 03:55:48,135 INFO [Listener at localhost/36131] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-27 03:55:48,135 INFO [Listener at localhost/36131] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-27 03:55:48,135 INFO [Listener at localhost/36131] regionserver.HRegion(2259): Starting compaction of 95b223459399eec298605c6ce5cd65f6/family1 in testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:48,135 INFO [Listener at localhost/36131] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/690f82499a7d4d4cb35fa108a740f436, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/61e706d4da9642848d106bb629cecf86, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/c2350ce460694fa180dafc621be33c0a, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/fb5bc9014897411faf086ce8adbe7351, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/21bfc35c25b84019b6fd753b46e8be23] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp, totalSize=27.1 K 2023-03-27 03:55:48,135 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 690f82499a7d4d4cb35fa108a740f436, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889340689 2023-03-27 03:55:48,136 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 61e706d4da9642848d106bb629cecf86, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889342224 2023-03-27 03:55:48,136 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting c2350ce460694fa180dafc621be33c0a, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889343393 2023-03-27 03:55:48,137 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting fb5bc9014897411faf086ce8adbe7351, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889345334 2023-03-27 03:55:48,137 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 21bfc35c25b84019b6fd753b46e8be23, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-27 03:55:48,146 INFO [Listener at localhost/36131] throttle.PressureAwareThroughputController(145): 95b223459399eec298605c6ce5cd65f6#family1#compaction#198 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:48,183 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303273930465516d14a8bb06f1fa835cfb324_del 2023-03-27 03:55:48,183 INFO [Listener at localhost/36131] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303273930465516d14a8bb06f1fa835cfb324_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303273930465516d14a8bb06f1fa835cfb324_del 2023-03-27 03:55:48,238 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 049d2f227abc4de0a4b986482e8dea88 2023-03-27 03:55:48,239 DEBUG [Listener at localhost/36131] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family1/049d2f227abc4de0a4b986482e8dea88 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/049d2f227abc4de0a4b986482e8dea88 2023-03-27 03:55:48,267 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 049d2f227abc4de0a4b986482e8dea88 2023-03-27 03:55:48,268 INFO [Listener at localhost/36131] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 95b223459399eec298605c6ce5cd65f6/family1 of 95b223459399eec298605c6ce5cd65f6 into 049d2f227abc4de0a4b986482e8dea88(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:55:48,269 DEBUG [Listener at localhost/36131] regionserver.HRegion(2289): Compaction status journal for 95b223459399eec298605c6ce5cd65f6: 2023-03-27 03:55:48,269 DEBUG [Listener at localhost/36131] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-27 03:55:48,269 DEBUG [Listener at localhost/36131] regionserver.HStore(1912): 95b223459399eec298605c6ce5cd65f6/family2 is initiating major compaction (all files) 2023-03-27 03:55:48,269 INFO [Listener at localhost/36131] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-27 03:55:48,269 INFO [Listener at localhost/36131] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-27 03:55:48,269 INFO [Listener at localhost/36131] regionserver.HRegion(2259): Starting compaction of 95b223459399eec298605c6ce5cd65f6/family2 in testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:48,269 INFO [Listener at localhost/36131] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/af6fe675c27f417d84fd72b8dd702508, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/35b9c7792772403cac8a65bbc4aa43fd, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/5513b324d73943e9842e2085f048d9cb, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/a1eb2586c1c24ca6bb042a425dc913e6, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/2ca56f349799475abc628515e864919c] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp, totalSize=25.4 K 2023-03-27 03:55:48,269 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting af6fe675c27f417d84fd72b8dd702508, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889340689 2023-03-27 03:55:48,270 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 35b9c7792772403cac8a65bbc4aa43fd, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889342224 2023-03-27 03:55:48,270 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 5513b324d73943e9842e2085f048d9cb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889343393 2023-03-27 03:55:48,270 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting a1eb2586c1c24ca6bb042a425dc913e6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889345334 2023-03-27 03:55:48,270 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 2ca56f349799475abc628515e864919c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-27 03:55:48,278 INFO [Listener at localhost/36131] throttle.PressureAwareThroughputController(145): 95b223459399eec298605c6ce5cd65f6#family2#compaction#199 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:48,297 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/690f82499a7d4d4cb35fa108a740f436, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/61e706d4da9642848d106bb629cecf86, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/c2350ce460694fa180dafc621be33c0a, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/fb5bc9014897411faf086ce8adbe7351, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/21bfc35c25b84019b6fd753b46e8be23] to archive 2023-03-27 03:55:48,298 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:48,299 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/690f82499a7d4d4cb35fa108a740f436 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/690f82499a7d4d4cb35fa108a740f436 2023-03-27 03:55:48,301 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/61e706d4da9642848d106bb629cecf86 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/61e706d4da9642848d106bb629cecf86 2023-03-27 03:55:48,301 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/c2350ce460694fa180dafc621be33c0a to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/c2350ce460694fa180dafc621be33c0a 2023-03-27 03:55:48,302 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/fb5bc9014897411faf086ce8adbe7351 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/fb5bc9014897411faf086ce8adbe7351 2023-03-27 03:55:48,303 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/21bfc35c25b84019b6fd753b46e8be23 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/21bfc35c25b84019b6fd753b46e8be23 2023-03-27 03:55:48,691 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303278d3fb3f6b24d4e678b2d1d0f44fefc6b_del 2023-03-27 03:55:48,692 INFO [Listener at localhost/36131] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202303278d3fb3f6b24d4e678b2d1d0f44fefc6b_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202303278d3fb3f6b24d4e678b2d1d0f44fefc6b_del 2023-03-27 03:55:49,108 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dba548027dfa4ae2990f5c44e7f59945 2023-03-27 03:55:49,108 DEBUG [Listener at localhost/36131] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/.tmp/family2/dba548027dfa4ae2990f5c44e7f59945 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/dba548027dfa4ae2990f5c44e7f59945 2023-03-27 03:55:49,111 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dba548027dfa4ae2990f5c44e7f59945 2023-03-27 03:55:49,112 INFO [Listener at localhost/36131] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 95b223459399eec298605c6ce5cd65f6/family2 of 95b223459399eec298605c6ce5cd65f6 into dba548027dfa4ae2990f5c44e7f59945(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:55:49,112 DEBUG [Listener at localhost/36131] regionserver.HRegion(2289): Compaction status journal for 95b223459399eec298605c6ce5cd65f6: 2023-03-27 03:55:49,113 DEBUG [Listener at localhost/36131] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-27 03:55:49,113 DEBUG [Listener at localhost/36131] regionserver.HStore(1912): ae7919e6ac686a9f3a2be09b01a0ff1a/family1 is initiating major compaction (all files) 2023-03-27 03:55:49,113 INFO [Listener at localhost/36131] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-27 03:55:49,113 INFO [Listener at localhost/36131] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-27 03:55:49,114 INFO [Listener at localhost/36131] regionserver.HRegion(2259): Starting compaction of ae7919e6ac686a9f3a2be09b01a0ff1a/family1 in testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:49,114 INFO [Listener at localhost/36131] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/b2a7d03d478c415591b3af0413c70978, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/e1e70a6f124a440ba818da5914b56d51, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/5deb15061f514a1c83724578fddecfcd, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/bdc559262daf42b7bca182ae7ef2a2b5, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/84eaf212dc844879bc2f8c7bdab0962b] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp, totalSize=27.1 K 2023-03-27 03:55:49,115 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting b2a7d03d478c415591b3af0413c70978, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889340689 2023-03-27 03:55:49,115 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting e1e70a6f124a440ba818da5914b56d51, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889342224 2023-03-27 03:55:49,116 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 5deb15061f514a1c83724578fddecfcd, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889343393 2023-03-27 03:55:49,116 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting bdc559262daf42b7bca182ae7ef2a2b5, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889345335 2023-03-27 03:55:49,116 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 84eaf212dc844879bc2f8c7bdab0962b, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-27 03:55:49,125 INFO [Listener at localhost/36131] throttle.PressureAwareThroughputController(145): ae7919e6ac686a9f3a2be09b01a0ff1a#family1#compaction#200 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:49,195 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/af6fe675c27f417d84fd72b8dd702508, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/35b9c7792772403cac8a65bbc4aa43fd, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/5513b324d73943e9842e2085f048d9cb, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/a1eb2586c1c24ca6bb042a425dc913e6, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/2ca56f349799475abc628515e864919c] to archive 2023-03-27 03:55:49,196 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:49,198 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/af6fe675c27f417d84fd72b8dd702508 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/af6fe675c27f417d84fd72b8dd702508 2023-03-27 03:55:49,199 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/35b9c7792772403cac8a65bbc4aa43fd to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/35b9c7792772403cac8a65bbc4aa43fd 2023-03-27 03:55:49,200 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/5513b324d73943e9842e2085f048d9cb to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/5513b324d73943e9842e2085f048d9cb 2023-03-27 03:55:49,202 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/a1eb2586c1c24ca6bb042a425dc913e6 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/a1eb2586c1c24ca6bb042a425dc913e6 2023-03-27 03:55:49,203 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/2ca56f349799475abc628515e864919c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family2/2ca56f349799475abc628515e864919c 2023-03-27 03:55:49,573 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303275224ce6c4fb94e11b70fc30d4fc161f2_del 2023-03-27 03:55:49,573 INFO [Listener at localhost/36131] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202303275224ce6c4fb94e11b70fc30d4fc161f2_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303275224ce6c4fb94e11b70fc30d4fc161f2_del 2023-03-27 03:55:49,626 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b115f333052946eaa7d3847cc7425977 2023-03-27 03:55:49,627 DEBUG [Listener at localhost/36131] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family1/b115f333052946eaa7d3847cc7425977 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/b115f333052946eaa7d3847cc7425977 2023-03-27 03:55:49,649 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b115f333052946eaa7d3847cc7425977 2023-03-27 03:55:49,650 INFO [Listener at localhost/36131] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in ae7919e6ac686a9f3a2be09b01a0ff1a/family1 of ae7919e6ac686a9f3a2be09b01a0ff1a into b115f333052946eaa7d3847cc7425977(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:55:49,650 DEBUG [Listener at localhost/36131] regionserver.HRegion(2289): Compaction status journal for ae7919e6ac686a9f3a2be09b01a0ff1a: 2023-03-27 03:55:49,650 DEBUG [Listener at localhost/36131] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-27 03:55:49,650 DEBUG [Listener at localhost/36131] regionserver.HStore(1912): ae7919e6ac686a9f3a2be09b01a0ff1a/family2 is initiating major compaction (all files) 2023-03-27 03:55:49,650 INFO [Listener at localhost/36131] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-27 03:55:49,650 INFO [Listener at localhost/36131] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-27 03:55:49,650 INFO [Listener at localhost/36131] regionserver.HRegion(2259): Starting compaction of ae7919e6ac686a9f3a2be09b01a0ff1a/family2 in testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:49,650 INFO [Listener at localhost/36131] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/0d554fb49a0c4852ab82d8a8b0fe2449, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/d68dc07e609d49b2820d6c42367cc852, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/6571508ba8654b42877a9f298086d0de, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/423ccbcd803544e0840e460a2130b169, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/7a1ae30890db498fa2fb7723b9df6fd7] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp, totalSize=25.4 K 2023-03-27 03:55:49,651 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 0d554fb49a0c4852ab82d8a8b0fe2449, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889340689 2023-03-27 03:55:49,651 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting d68dc07e609d49b2820d6c42367cc852, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889342224 2023-03-27 03:55:49,651 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 6571508ba8654b42877a9f298086d0de, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889343393 2023-03-27 03:55:49,652 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 423ccbcd803544e0840e460a2130b169, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889345335 2023-03-27 03:55:49,652 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 7a1ae30890db498fa2fb7723b9df6fd7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-27 03:55:49,666 INFO [Listener at localhost/36131] throttle.PressureAwareThroughputController(145): ae7919e6ac686a9f3a2be09b01a0ff1a#family2#compaction#201 average throughput is 0.12 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:49,683 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230327568e3b1fc57d451892063ffd841d6b65_del 2023-03-27 03:55:49,684 INFO [Listener at localhost/36131] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230327568e3b1fc57d451892063ffd841d6b65_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230327568e3b1fc57d451892063ffd841d6b65_del 2023-03-27 03:55:49,691 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/b2a7d03d478c415591b3af0413c70978, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/e1e70a6f124a440ba818da5914b56d51, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/5deb15061f514a1c83724578fddecfcd, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/bdc559262daf42b7bca182ae7ef2a2b5, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/84eaf212dc844879bc2f8c7bdab0962b] to archive 2023-03-27 03:55:49,692 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:49,693 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/b2a7d03d478c415591b3af0413c70978 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/b2a7d03d478c415591b3af0413c70978 2023-03-27 03:55:49,694 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/e1e70a6f124a440ba818da5914b56d51 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/e1e70a6f124a440ba818da5914b56d51 2023-03-27 03:55:49,695 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/5deb15061f514a1c83724578fddecfcd to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/5deb15061f514a1c83724578fddecfcd 2023-03-27 03:55:49,696 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/bdc559262daf42b7bca182ae7ef2a2b5 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/bdc559262daf42b7bca182ae7ef2a2b5 2023-03-27 03:55:49,697 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/84eaf212dc844879bc2f8c7bdab0962b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/84eaf212dc844879bc2f8c7bdab0962b 2023-03-27 03:55:50,503 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 58e1fae69a6d4ee08c0f8fe159044f07 2023-03-27 03:55:50,504 DEBUG [Listener at localhost/36131] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/.tmp/family2/58e1fae69a6d4ee08c0f8fe159044f07 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/58e1fae69a6d4ee08c0f8fe159044f07 2023-03-27 03:55:50,508 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 58e1fae69a6d4ee08c0f8fe159044f07 2023-03-27 03:55:50,509 INFO [Listener at localhost/36131] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in ae7919e6ac686a9f3a2be09b01a0ff1a/family2 of ae7919e6ac686a9f3a2be09b01a0ff1a into 58e1fae69a6d4ee08c0f8fe159044f07(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:55:50,509 DEBUG [Listener at localhost/36131] regionserver.HRegion(2289): Compaction status journal for ae7919e6ac686a9f3a2be09b01a0ff1a: 2023-03-27 03:55:50,509 DEBUG [Listener at localhost/36131] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-27 03:55:50,509 DEBUG [Listener at localhost/36131] regionserver.HStore(1912): d0bf0165da69488ffc28ab9679dbd8e9/family1 is initiating major compaction (all files) 2023-03-27 03:55:50,510 INFO [Listener at localhost/36131] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-27 03:55:50,510 INFO [Listener at localhost/36131] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-27 03:55:50,511 INFO [Listener at localhost/36131] regionserver.HRegion(2259): Starting compaction of d0bf0165da69488ffc28ab9679dbd8e9/family1 in testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:50,511 INFO [Listener at localhost/36131] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/1258ec316a744477bf94485bf4ff163d, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/534e0b6bdbfe47c59e6f128d45748c06, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/889fd089a0e54b0aae35e55caeb9790b, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/090c02539bea4b2885550efe2dc12cce, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/ebbb196f8d8945b9aa4b100a7c5a57ea] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp, totalSize=27.1 K 2023-03-27 03:55:50,511 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 1258ec316a744477bf94485bf4ff163d, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889340690 2023-03-27 03:55:50,511 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 534e0b6bdbfe47c59e6f128d45748c06, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889342224 2023-03-27 03:55:50,512 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 889fd089a0e54b0aae35e55caeb9790b, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889343394 2023-03-27 03:55:50,512 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 090c02539bea4b2885550efe2dc12cce, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889345335 2023-03-27 03:55:50,513 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting ebbb196f8d8945b9aa4b100a7c5a57ea, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-27 03:55:50,523 INFO [Listener at localhost/36131] throttle.PressureAwareThroughputController(145): d0bf0165da69488ffc28ab9679dbd8e9#family1#compaction#202 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:50,597 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/0d554fb49a0c4852ab82d8a8b0fe2449, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/d68dc07e609d49b2820d6c42367cc852, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/6571508ba8654b42877a9f298086d0de, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/423ccbcd803544e0840e460a2130b169, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/7a1ae30890db498fa2fb7723b9df6fd7] to archive 2023-03-27 03:55:50,597 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:50,599 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/0d554fb49a0c4852ab82d8a8b0fe2449 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/0d554fb49a0c4852ab82d8a8b0fe2449 2023-03-27 03:55:50,600 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/d68dc07e609d49b2820d6c42367cc852 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/d68dc07e609d49b2820d6c42367cc852 2023-03-27 03:55:50,604 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/6571508ba8654b42877a9f298086d0de to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/6571508ba8654b42877a9f298086d0de 2023-03-27 03:55:50,606 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/423ccbcd803544e0840e460a2130b169 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/423ccbcd803544e0840e460a2130b169 2023-03-27 03:55:50,607 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/7a1ae30890db498fa2fb7723b9df6fd7 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family2/7a1ae30890db498fa2fb7723b9df6fd7 2023-03-27 03:55:50,969 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230327a600c883ee964e13aec361186302ff7c_del 2023-03-27 03:55:50,969 INFO [Listener at localhost/36131] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230327a600c883ee964e13aec361186302ff7c_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230327a600c883ee964e13aec361186302ff7c_del 2023-03-27 03:55:51,034 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a434878e9d0a4a68bacd8ce24fb2f432 2023-03-27 03:55:51,034 DEBUG [Listener at localhost/36131] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family1/a434878e9d0a4a68bacd8ce24fb2f432 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/a434878e9d0a4a68bacd8ce24fb2f432 2023-03-27 03:55:51,069 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a434878e9d0a4a68bacd8ce24fb2f432 2023-03-27 03:55:51,071 INFO [Listener at localhost/36131] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d0bf0165da69488ffc28ab9679dbd8e9/family1 of d0bf0165da69488ffc28ab9679dbd8e9 into a434878e9d0a4a68bacd8ce24fb2f432(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-03-27 03:55:51,071 DEBUG [Listener at localhost/36131] regionserver.HRegion(2289): Compaction status journal for d0bf0165da69488ffc28ab9679dbd8e9: 2023-03-27 03:55:51,071 DEBUG [Listener at localhost/36131] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-03-27 03:55:51,071 DEBUG [Listener at localhost/36131] regionserver.HStore(1912): d0bf0165da69488ffc28ab9679dbd8e9/family2 is initiating major compaction (all files) 2023-03-27 03:55:51,071 INFO [Listener at localhost/36131] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-03-27 03:55:51,071 INFO [Listener at localhost/36131] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-03-27 03:55:51,071 INFO [Listener at localhost/36131] regionserver.HRegion(2259): Starting compaction of d0bf0165da69488ffc28ab9679dbd8e9/family2 in testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:51,072 INFO [Listener at localhost/36131] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/8289773ba559468fadc4e4bd8d7251eb, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/18ecda24e0de4a0e93ba368697161804, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/a1c2a153dcbc4b67a6b61e83dfd4d27b, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/982d8a5c6b544a91a61cfc38abfaecdb, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/7f470426775349f5bbe2a5b10ce13af3] into tmpdir=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp, totalSize=25.4 K 2023-03-27 03:55:51,072 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 8289773ba559468fadc4e4bd8d7251eb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1679889340690 2023-03-27 03:55:51,073 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 18ecda24e0de4a0e93ba368697161804, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1679889342224 2023-03-27 03:55:51,073 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting a1c2a153dcbc4b67a6b61e83dfd4d27b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1679889343394 2023-03-27 03:55:51,074 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 982d8a5c6b544a91a61cfc38abfaecdb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1679889345335 2023-03-27 03:55:51,074 DEBUG [Listener at localhost/36131] compactions.Compactor(207): Compacting 7f470426775349f5bbe2a5b10ce13af3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-03-27 03:55:51,085 INFO [Listener at localhost/36131] throttle.PressureAwareThroughputController(145): d0bf0165da69488ffc28ab9679dbd8e9#family2#compaction#203 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-03-27 03:55:51,090 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/1258ec316a744477bf94485bf4ff163d, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/534e0b6bdbfe47c59e6f128d45748c06, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/889fd089a0e54b0aae35e55caeb9790b, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/090c02539bea4b2885550efe2dc12cce, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/ebbb196f8d8945b9aa4b100a7c5a57ea] to archive 2023-03-27 03:55:51,091 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:51,092 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/1258ec316a744477bf94485bf4ff163d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/1258ec316a744477bf94485bf4ff163d 2023-03-27 03:55:51,094 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/534e0b6bdbfe47c59e6f128d45748c06 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/534e0b6bdbfe47c59e6f128d45748c06 2023-03-27 03:55:51,095 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/889fd089a0e54b0aae35e55caeb9790b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/889fd089a0e54b0aae35e55caeb9790b 2023-03-27 03:55:51,096 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/090c02539bea4b2885550efe2dc12cce to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/090c02539bea4b2885550efe2dc12cce 2023-03-27 03:55:51,097 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/ebbb196f8d8945b9aa4b100a7c5a57ea to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/ebbb196f8d8945b9aa4b100a7c5a57ea 2023-03-27 03:55:51,493 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023032771347d80e3ee48a4b295ad7532e46981_del 2023-03-27 03:55:51,493 INFO [Listener at localhost/36131] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023032771347d80e3ee48a4b295ad7532e46981_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023032771347d80e3ee48a4b295ad7532e46981_del 2023-03-27 03:55:52,308 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 308c3eb5a6894ce4bc68b46e62ec3928 2023-03-27 03:55:52,309 DEBUG [Listener at localhost/36131] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/.tmp/family2/308c3eb5a6894ce4bc68b46e62ec3928 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/308c3eb5a6894ce4bc68b46e62ec3928 2023-03-27 03:55:52,312 INFO [Listener at localhost/36131] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 308c3eb5a6894ce4bc68b46e62ec3928 2023-03-27 03:55:52,314 INFO [Listener at localhost/36131] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in d0bf0165da69488ffc28ab9679dbd8e9/family2 of d0bf0165da69488ffc28ab9679dbd8e9 into 308c3eb5a6894ce4bc68b46e62ec3928(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-03-27 03:55:52,314 DEBUG [Listener at localhost/36131] regionserver.HRegion(2289): Compaction status journal for d0bf0165da69488ffc28ab9679dbd8e9: 2023-03-27 03:55:52,319 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-27 03:55:52,322 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-27 03:55:52,327 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-27 03:55:52,327 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-27 03:55:52,332 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38271] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xCB7\xB8\xC1\xDF\xD3\xE90\xD2J\xC9\xD8lDb\xF3"\x10\x0D9,\xD6\xC2+]HP%\xFC2:\xDF<6*\x106\x85\xD2\xB4\xD2\xB2{\xD8\x026\xF5\xC6Q.e.2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-03-27 03:55:52,355 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-27 03:55:52,356 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-27 03:55:52,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:55:52,357 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1679889952357 2023-03-27 03:55:52,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-03-27 03:55:52,389 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/8289773ba559468fadc4e4bd8d7251eb, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/18ecda24e0de4a0e93ba368697161804, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/a1c2a153dcbc4b67a6b61e83dfd4d27b, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/982d8a5c6b544a91a61cfc38abfaecdb, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/7f470426775349f5bbe2a5b10ce13af3] to archive 2023-03-27 03:55:52,390 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:52,391 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/8289773ba559468fadc4e4bd8d7251eb to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/8289773ba559468fadc4e4bd8d7251eb 2023-03-27 03:55:52,392 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/18ecda24e0de4a0e93ba368697161804 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/18ecda24e0de4a0e93ba368697161804 2023-03-27 03:55:52,393 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/a1c2a153dcbc4b67a6b61e83dfd4d27b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/a1c2a153dcbc4b67a6b61e83dfd4d27b 2023-03-27 03:55:52,394 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/982d8a5c6b544a91a61cfc38abfaecdb to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/982d8a5c6b544a91a61cfc38abfaecdb 2023-03-27 03:55:52,395 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/7f470426775349f5bbe2a5b10ce13af3 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family2/7f470426775349f5bbe2a5b10ce13af3 2023-03-27 03:55:52,786 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-03-27 03:55:52,810 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202303273930465516d14a8bb06f1fa835cfb324_del 2023-03-27 03:55:52,835 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202303275224ce6c4fb94e11b70fc30d4fc161f2_del 2023-03-27 03:55:52,870 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230327a600c883ee964e13aec361186302ff7c_del 2023-03-27 03:55:52,870 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-27 03:55:52,871 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] zookeeper.ReadOnlyZKClient(139): Connect 0x4a433e5e to 127.0.0.1:62246 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-03-27 03:55:52,902 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@47d8ef00, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-03-27 03:55:52,902 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230327 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-27 03:55:53,090 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230327ba7ae6333bb24a4da80da72b3fc0f55d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230327ba7ae6333bb24a4da80da72b3fc0f55d 2023-03-27 03:55:53,092 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230327 table='testMajorCompactionFromAdmin' column='family1' 2023-03-27 03:55:53,105 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-03-27 03:55:53,108 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45908, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-03-27 03:55:53,117 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-03-27 03:55:53,119 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55474, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-03-27 03:55:53,173 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230327/family1/635c3bd56f6d4c2495a2c60240cb98bf first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-03-27 03:55:53,175 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230327/family1/635c3bd56f6d4c2495a2c60240cb98bf}] 2023-03-27 03:55:53,186 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230327/family1/635c3bd56f6d4c2495a2c60240cb98bf for inclusion in 95b223459399eec298605c6ce5cd65f6/family1 2023-03-27 03:55:53,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-03-27 03:55:53,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first= last=1 2023-03-27 03:55:53,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for 95b223459399eec298605c6ce5cd65f6: 2023-03-27 03:55:53,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230327/family1/635c3bd56f6d4c2495a2c60240cb98bf to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionFromAdmin__3t7s5feers2lceguht1beili464tglicckepshksqqretj0bjqtvf09u7an3rusf/family1/635c3bd56f6d4c2495a2c60240cb98bf 2023-03-27 03:55:53,238 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionFromAdmin__3t7s5feers2lceguht1beili464tglicckepshksqqretj0bjqtvf09u7an3rusf/family1/635c3bd56f6d4c2495a2c60240cb98bf as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/fd70a95cad904777a77a193e7ea9148e_SeqId_28_ 2023-03-27 03:55:53,239 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionFromAdmin__3t7s5feers2lceguht1beili464tglicckepshksqqretj0bjqtvf09u7an3rusf/family1/635c3bd56f6d4c2495a2c60240cb98bf into 95b223459399eec298605c6ce5cd65f6/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/fd70a95cad904777a77a193e7ea9148e_SeqId_28_ - updating store file list. 2023-03-27 03:55:53,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/fd70a95cad904777a77a193e7ea9148e_SeqId_28_ into 95b223459399eec298605c6ce5cd65f6/family1 2023-03-27 03:55:53,275 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionFromAdmin__3t7s5feers2lceguht1beili464tglicckepshksqqretj0bjqtvf09u7an3rusf/family1/635c3bd56f6d4c2495a2c60240cb98bf into 95b223459399eec298605c6ce5cd65f6/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/family1/fd70a95cad904777a77a193e7ea9148e_SeqId_28_) 2023-03-27 03:55:53,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionFromAdmin__3t7s5feers2lceguht1beili464tglicckepshksqqretj0bjqtvf09u7an3rusf/family1/635c3bd56f6d4c2495a2c60240cb98bf 2023-03-27 03:55:53,278 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230327 table='testMajorCompactionFromAdmin' column='family1' 2023-03-27 03:55:53,278 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:53,281 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023032745c61f79664b4b7fb6674a9df6d9ca7d to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023032745c61f79664b4b7fb6674a9df6d9ca7d 2023-03-27 03:55:53,282 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230327b29087d566ef4b7bae7cc7e09d26c8ba to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230327b29087d566ef4b7bae7cc7e09d26c8ba 2023-03-27 03:55:53,283 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230327d315634bec454bca99b62789046d98ec to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230327d315634bec454bca99b62789046d98ec 2023-03-27 03:55:53,284 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230327fa5cc888337540ef8088eaf94ce0023f to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230327fa5cc888337540ef8088eaf94ce0023f 2023-03-27 03:55:53,284 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-27 03:55:53,284 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230327' 2023-03-27 03:55:53,285 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230327 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-27 03:55:53,285 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230327 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-27 03:55:53,878 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230327bb34f19b6fdb41118a8bf1099a26695b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230327bb34f19b6fdb41118a8bf1099a26695b 2023-03-27 03:55:53,879 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230327 table='testMajorCompactionFromAdmin' column='family1' 2023-03-27 03:55:53,941 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230327/family1/989fc2400395493f9bd95c0740c26f40 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-03-27 03:55:53,942 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230327/family1/989fc2400395493f9bd95c0740c26f40}] 2023-03-27 03:55:53,945 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230327/family1/989fc2400395493f9bd95c0740c26f40 for inclusion in d0bf0165da69488ffc28ab9679dbd8e9/family1 2023-03-27 03:55:53,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-03-27 03:55:53,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first=2 last= 2023-03-27 03:55:53,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for d0bf0165da69488ffc28ab9679dbd8e9: 2023-03-27 03:55:53,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230327/family1/989fc2400395493f9bd95c0740c26f40 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionFromAdmin__mlu6pa4219j0rtjb4854cqlnd59cfjcamq5egbh4fnitfst51gtlk1b9n26f5bns/family1/989fc2400395493f9bd95c0740c26f40 2023-03-27 03:55:53,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionFromAdmin__mlu6pa4219j0rtjb4854cqlnd59cfjcamq5egbh4fnitfst51gtlk1b9n26f5bns/family1/989fc2400395493f9bd95c0740c26f40 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/ecd646c0a50c4cd195e0539477d0b48b_SeqId_28_ 2023-03-27 03:55:53,970 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionFromAdmin__mlu6pa4219j0rtjb4854cqlnd59cfjcamq5egbh4fnitfst51gtlk1b9n26f5bns/family1/989fc2400395493f9bd95c0740c26f40 into d0bf0165da69488ffc28ab9679dbd8e9/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/ecd646c0a50c4cd195e0539477d0b48b_SeqId_28_ - updating store file list. 2023-03-27 03:55:53,991 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/ecd646c0a50c4cd195e0539477d0b48b_SeqId_28_ into d0bf0165da69488ffc28ab9679dbd8e9/family1 2023-03-27 03:55:53,991 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionFromAdmin__mlu6pa4219j0rtjb4854cqlnd59cfjcamq5egbh4fnitfst51gtlk1b9n26f5bns/family1/989fc2400395493f9bd95c0740c26f40 into d0bf0165da69488ffc28ab9679dbd8e9/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/family1/ecd646c0a50c4cd195e0539477d0b48b_SeqId_28_) 2023-03-27 03:55:53,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionFromAdmin__mlu6pa4219j0rtjb4854cqlnd59cfjcamq5egbh4fnitfst51gtlk1b9n26f5bns/family1/989fc2400395493f9bd95c0740c26f40 2023-03-27 03:55:53,993 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230327 table='testMajorCompactionFromAdmin' column='family1' 2023-03-27 03:55:53,994 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:53,996 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303273593cf738f004068a64c899f122fe939 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202303273593cf738f004068a64c899f122fe939 2023-03-27 03:55:53,997 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230327b721cdc2ad734e98a633e281bf0f263f to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230327b721cdc2ad734e98a633e281bf0f263f 2023-03-27 03:55:53,997 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230327c6dabeb9f1bb442c95bb45971deadead to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230327c6dabeb9f1bb442c95bb45971deadead 2023-03-27 03:55:53,998 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230327eba97634e826417b9cc6aa8522c4bc0f to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230327eba97634e826417b9cc6aa8522c4bc0f 2023-03-27 03:55:53,998 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-27 03:55:53,998 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230327' 2023-03-27 03:55:53,999 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230327 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-27 03:55:53,999 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230327 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-27 03:55:54,198 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023032740daa72223c84fd59856a9a6c3cc013e to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032740daa72223c84fd59856a9a6c3cc013e 2023-03-27 03:55:54,199 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230327 table='testMajorCompactionFromAdmin' column='family1' 2023-03-27 03:55:54,257 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230327/family1/8c91928fdf974b9cb84b020b0ca5f4c1 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-03-27 03:55:54,258 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a., hostname=jenkins-hbase4.apache.org,39727,1679889218560, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230327/family1/8c91928fdf974b9cb84b020b0ca5f4c1}] 2023-03-27 03:55:54,261 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(763): Validating hfile at hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230327/family1/8c91928fdf974b9cb84b020b0ca5f4c1 for inclusion in ae7919e6ac686a9f3a2be09b01a0ff1a/family1 2023-03-27 03:55:54,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-03-27 03:55:54,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(777): Region bounds: first=1 last=2 2023-03-27 03:55:54,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegion(2446): Flush status journal for ae7919e6ac686a9f3a2be09b01a0ff1a: 2023-03-27 03:55:54,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230327/family1/8c91928fdf974b9cb84b020b0ca5f4c1 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionFromAdmin__pbqhaer2fbbaor1c9ot6eecd6mu2v44td71la0lqc5d5sl7m8klk6u7fgotm0c1p/family1/8c91928fdf974b9cb84b020b0ca5f4c1 2023-03-27 03:55:54,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionFromAdmin__pbqhaer2fbbaor1c9ot6eecd6mu2v44td71la0lqc5d5sl7m8klk6u7fgotm0c1p/family1/8c91928fdf974b9cb84b020b0ca5f4c1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/31e1c5344d4640ccbfc22c8c7ba564bd_SeqId_28_ 2023-03-27 03:55:54,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(849): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionFromAdmin__pbqhaer2fbbaor1c9ot6eecd6mu2v44td71la0lqc5d5sl7m8klk6u7fgotm0c1p/family1/8c91928fdf974b9cb84b020b0ca5f4c1 into ae7919e6ac686a9f3a2be09b01a0ff1a/family1 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/31e1c5344d4640ccbfc22c8c7ba564bd_SeqId_28_ - updating store file list. 2023-03-27 03:55:54,320 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(882): Loaded HFile hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/31e1c5344d4640ccbfc22c8c7ba564bd_SeqId_28_ into ae7919e6ac686a9f3a2be09b01a0ff1a/family1 2023-03-27 03:55:54,320 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.HStore(855): Successfully loaded hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionFromAdmin__pbqhaer2fbbaor1c9ot6eecd6mu2v44td71la0lqc5d5sl7m8klk6u7fgotm0c1p/family1/8c91928fdf974b9cb84b020b0ca5f4c1 into ae7919e6ac686a9f3a2be09b01a0ff1a/family1 (new location: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/family1/31e1c5344d4640ccbfc22c8c7ba564bd_SeqId_28_) 2023-03-27 03:55:54,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39727] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/staging/jenkins__testMajorCompactionFromAdmin__pbqhaer2fbbaor1c9ot6eecd6mu2v44td71la0lqc5d5sl7m8klk6u7fgotm0c1p/family1/8c91928fdf974b9cb84b020b0ca5f4c1 2023-03-27 03:55:54,322 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230327 table='testMajorCompactionFromAdmin' column='family1' 2023-03-27 03:55:54,323 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:54,324 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230327174b3b347db6493db21425cfa5754d9c to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230327174b3b347db6493db21425cfa5754d9c 2023-03-27 03:55:54,325 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032777f94cb8477b4c24a0cc2b6370a8be5b to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032777f94cb8477b4c24a0cc2b6370a8be5b 2023-03-27 03:55:54,326 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303277c93768bbd0e4f3a972eb9c770bf9b45 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303277c93768bbd0e4f3a972eb9c770bf9b45 2023-03-27 03:55:54,327 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032798398f24982c45308ef8827e1e50ac21 to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023032798398f24982c45308ef8827e1e50ac21 2023-03-27 03:55:54,327 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-03-27 03:55:54,327 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230327' 2023-03-27 03:55:54,327 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230327 for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-27 03:55:54,328 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-03-27 03:55:54,328 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-03-27 03:55:54,328 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(367): [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303273930465516d14a8bb06f1fa835cfb324_del] 2023-03-27 03:55:54,328 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:54,329 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303273930465516d14a8bb06f1fa835cfb324_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202303273930465516d14a8bb06f1fa835cfb324_del 2023-03-27 03:55:54,330 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-27 03:55:54,330 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(367): [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303275224ce6c4fb94e11b70fc30d4fc161f2_del] 2023-03-27 03:55:54,330 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:54,331 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303275224ce6c4fb94e11b70fc30d4fc161f2_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202303275224ce6c4fb94e11b70fc30d4fc161f2_del 2023-03-27 03:55:54,331 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-27 03:55:54,331 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] compactions.PartitionedMobCompactor(367): [hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230327a600c883ee964e13aec361186302ff7c_del] 2023-03-27 03:55:54,331 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] backup.HFileArchiver(360): Archiving compacted files. 2023-03-27 03:55:54,333 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230327a600c883ee964e13aec361186302ff7c_del to hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230327a600c883ee964e13aec361186302ff7c_del 2023-03-27 03:55:54,333 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-03-27 03:55:54,333 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1679889222078] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-03-27 03:55:54,336 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-03-27 03:55:54,337 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 1.9810 sec 2023-03-27 03:55:54,354 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-27 03:55:54,356 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-27 03:55:54,359 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-03-27 03:55:54,360 DEBUG [Listener at localhost/36131] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-03-27 03:55:54,366 INFO [Listener at localhost/36131] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-03-27 03:55:54,416 INFO [Listener at localhost/36131] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=250 (was 255), OpenFileDescriptor=639 (was 642), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=392 (was 391) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=8582 (was 8613) 2023-03-27 03:55:54,416 DEBUG [Listener at localhost/36131] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x69d78ff9 to 127.0.0.1:62246 2023-03-27 03:55:54,416 DEBUG [Listener at localhost/36131] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:55:54,417 INFO [Listener at localhost/36131] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-03-27 03:55:54,417 INFO [Listener at localhost/36131] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-03-27 03:55:54,417 DEBUG [Listener at localhost/36131] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x18f0808a to 127.0.0.1:62246 2023-03-27 03:55:54,417 DEBUG [Listener at localhost/36131] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:55:54,418 DEBUG [Listener at localhost/36131] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-03-27 03:55:54,418 DEBUG [Listener at localhost/36131] util.JVMClusterUtil(257): Found active master hash=688687638, stopped=false 2023-03-27 03:55:54,418 INFO [Listener at localhost/36131] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,38271,1679889217353 2023-03-27 03:55:54,420 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-27 03:55:54,420 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-03-27 03:55:54,421 INFO [Listener at localhost/36131] procedure2.ProcedureExecutor(629): Stopping 2023-03-27 03:55:54,421 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-27 03:55:54,421 DEBUG [Listener at localhost/36131] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5a340601 to 127.0.0.1:62246 2023-03-27 03:55:54,421 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-27 03:55:54,421 DEBUG [Listener at localhost/36131] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:55:54,421 INFO [Listener at localhost/36131] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,39727,1679889218560' ***** 2023-03-27 03:55:54,422 INFO [Listener at localhost/36131] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-03-27 03:55:54,422 INFO [RS:0;jenkins-hbase4:39727] regionserver.HeapMemoryManager(220): Stopping 2023-03-27 03:55:54,422 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-03-27 03:55:54,422 INFO [RS:0;jenkins-hbase4:39727] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-03-27 03:55:54,422 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-03-27 03:55:54,422 INFO [RS:0;jenkins-hbase4:39727] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-03-27 03:55:54,422 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(3304): Received CLOSE for 78fc96d1b1d88244802e871a9fbedaf3 2023-03-27 03:55:54,423 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(3304): Received CLOSE for eb204466139b909fd8d143a4c79d91ad 2023-03-27 03:55:54,423 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(3304): Received CLOSE for a03c85498f16edb6952ae7ec4494cfbd 2023-03-27 03:55:54,423 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 78fc96d1b1d88244802e871a9fbedaf3, disabling compactions & flushes 2023-03-27 03:55:54,423 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(3304): Received CLOSE for 7bab4d71becda434aedcb5da6cc18929 2023-03-27 03:55:54,423 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:55:54,423 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(3304): Received CLOSE for 9ce98af924f76608e57db6d1e9c69a22 2023-03-27 03:55:54,423 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(3304): Received CLOSE for 95b223459399eec298605c6ce5cd65f6 2023-03-27 03:55:54,423 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(3304): Received CLOSE for 08f8d4ce92d3daa28bf960c05f8fc02f 2023-03-27 03:55:54,423 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(3304): Received CLOSE for 72b74f7fa0bbf3322278037ab0d1d7f8 2023-03-27 03:55:54,423 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(3304): Received CLOSE for ae7919e6ac686a9f3a2be09b01a0ff1a 2023-03-27 03:55:54,423 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(3304): Received CLOSE for f2819f01ee4f081befffef519bfab5a6 2023-03-27 03:55:54,423 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(3304): Received CLOSE for faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:55:54,423 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(3304): Received CLOSE for 647148480c2004d294d48558b731b404 2023-03-27 03:55:54,423 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(3304): Received CLOSE for 08cac3032353cd1d586bb4d48f4a89b2 2023-03-27 03:55:54,423 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(3304): Received CLOSE for 1002361bf1e6585024f86241020836f1 2023-03-27 03:55:54,423 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(3304): Received CLOSE for c0982fa95f3b2e90e5d67d21d9f32707 2023-03-27 03:55:54,423 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(3304): Received CLOSE for 1b540ab2fb7dd087d0e66746ccea8753 2023-03-27 03:55:54,423 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(3304): Received CLOSE for a7921456431e437526c5b87521696821 2023-03-27 03:55:54,423 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(3304): Received CLOSE for d0bf0165da69488ffc28ab9679dbd8e9 2023-03-27 03:55:54,423 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(3304): Received CLOSE for a96b2587b7874c72fc83ce1239d003a2 2023-03-27 03:55:54,424 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(3304): Received CLOSE for 5f700223782e72e7df2f686db6ddd489 2023-03-27 03:55:54,424 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:54,424 DEBUG [RS:0;jenkins-hbase4:39727] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5c4e655e to 127.0.0.1:62246 2023-03-27 03:55:54,423 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:55:54,424 DEBUG [RS:0;jenkins-hbase4:39727] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:55:54,424 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. after waiting 0 ms 2023-03-27 03:55:54,424 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:55:54,425 INFO [RS:0;jenkins-hbase4:39727] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-03-27 03:55:54,425 INFO [RS:0;jenkins-hbase4:39727] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-03-27 03:55:54,425 INFO [RS:0;jenkins-hbase4:39727] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-03-27 03:55:54,425 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-03-27 03:55:54,425 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-03-27 03:55:54,425 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-03-27 03:55:54,426 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-03-27 03:55:54,426 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-03-27 03:55:54,426 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-03-27 03:55:54,426 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-03-27 03:55:54,426 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-03-27 03:55:54,426 DEBUG [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(1505): Waiting on 08cac3032353cd1d586bb4d48f4a89b2, 08f8d4ce92d3daa28bf960c05f8fc02f, 1002361bf1e6585024f86241020836f1, 1588230740, 1b540ab2fb7dd087d0e66746ccea8753, 5f700223782e72e7df2f686db6ddd489, 647148480c2004d294d48558b731b404, 72b74f7fa0bbf3322278037ab0d1d7f8, 78fc96d1b1d88244802e871a9fbedaf3, 7bab4d71becda434aedcb5da6cc18929, 95b223459399eec298605c6ce5cd65f6, 9ce98af924f76608e57db6d1e9c69a22, a03c85498f16edb6952ae7ec4494cfbd, a7921456431e437526c5b87521696821, a96b2587b7874c72fc83ce1239d003a2, ae7919e6ac686a9f3a2be09b01a0ff1a, c0982fa95f3b2e90e5d67d21d9f32707, d0bf0165da69488ffc28ab9679dbd8e9, eb204466139b909fd8d143a4c79d91ad, f2819f01ee4f081befffef519bfab5a6, faf5e912b9b1c732d5f1903dc1242b64 2023-03-27 03:55:54,430 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithMonthlyPolicy/78fc96d1b1d88244802e871a9fbedaf3/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-27 03:55:54,431 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:55:54,431 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 78fc96d1b1d88244802e871a9fbedaf3: 2023-03-27 03:55:54,431 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-03-27 03:55:54,431 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1679889233238.78fc96d1b1d88244802e871a9fbedaf3. 2023-03-27 03:55:54,431 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing eb204466139b909fd8d143a4c79d91ad, disabling compactions & flushes 2023-03-27 03:55:54,432 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:55:54,432 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:55:54,432 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. after waiting 0 ms 2023-03-27 03:55:54,432 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:55:54,441 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithWeeklyPolicy/eb204466139b909fd8d143a4c79d91ad/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-27 03:55:54,442 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:55:54,442 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for eb204466139b909fd8d143a4c79d91ad: 2023-03-27 03:55:54,442 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1679889241853.eb204466139b909fd8d143a4c79d91ad. 2023-03-27 03:55:54,442 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a03c85498f16edb6952ae7ec4494cfbd, disabling compactions & flushes 2023-03-27 03:55:54,443 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:55:54,443 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:55:54,443 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. after waiting 0 ms 2023-03-27 03:55:54,443 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:55:54,445 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/a03c85498f16edb6952ae7ec4494cfbd/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-27 03:55:54,446 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:55:54,446 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a03c85498f16edb6952ae7ec4494cfbd: 2023-03-27 03:55:54,446 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1679889261295.a03c85498f16edb6952ae7ec4494cfbd. 2023-03-27 03:55:54,446 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7bab4d71becda434aedcb5da6cc18929, disabling compactions & flushes 2023-03-27 03:55:54,446 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:54,446 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:54,446 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. after waiting 0 ms 2023-03-27 03:55:54,446 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:54,458 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/7bab4d71becda434aedcb5da6cc18929/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-27 03:55:54,459 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:54,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7bab4d71becda434aedcb5da6cc18929: 2023-03-27 03:55:54,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1679889316507.7bab4d71becda434aedcb5da6cc18929. 2023-03-27 03:55:54,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 9ce98af924f76608e57db6d1e9c69a22, disabling compactions & flushes 2023-03-27 03:55:54,459 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:54,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:54,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. after waiting 0 ms 2023-03-27 03:55:54,459 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:54,463 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/9ce98af924f76608e57db6d1e9c69a22/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-27 03:55:54,464 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:54,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 9ce98af924f76608e57db6d1e9c69a22: 2023-03-27 03:55:54,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1679889316507.9ce98af924f76608e57db6d1e9c69a22. 2023-03-27 03:55:54,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 95b223459399eec298605c6ce5cd65f6, disabling compactions & flushes 2023-03-27 03:55:54,464 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:54,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:54,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. after waiting 0 ms 2023-03-27 03:55:54,464 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:54,467 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/95b223459399eec298605c6ce5cd65f6/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-27 03:55:54,468 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:54,468 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 95b223459399eec298605c6ce5cd65f6: 2023-03-27 03:55:54,468 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1679889338545.95b223459399eec298605c6ce5cd65f6. 2023-03-27 03:55:54,468 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 08f8d4ce92d3daa28bf960c05f8fc02f, disabling compactions & flushes 2023-03-27 03:55:54,468 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f. 2023-03-27 03:55:54,468 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f. 2023-03-27 03:55:54,468 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f. after waiting 0 ms 2023-03-27 03:55:54,468 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f. 2023-03-27 03:55:54,468 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 08f8d4ce92d3daa28bf960c05f8fc02f 1/1 column families, dataSize=109 B heapSize=592 B 2023-03-27 03:55:54,474 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/namespace/08f8d4ce92d3daa28bf960c05f8fc02f/.tmp/info/d47a6bb42a7a480e9692d29f3ab9d78e 2023-03-27 03:55:54,478 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/namespace/08f8d4ce92d3daa28bf960c05f8fc02f/.tmp/info/d47a6bb42a7a480e9692d29f3ab9d78e as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/namespace/08f8d4ce92d3daa28bf960c05f8fc02f/info/d47a6bb42a7a480e9692d29f3ab9d78e 2023-03-27 03:55:54,483 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/namespace/08f8d4ce92d3daa28bf960c05f8fc02f/info/d47a6bb42a7a480e9692d29f3ab9d78e, entries=3, sequenceid=7, filesize=4.8 K 2023-03-27 03:55:54,484 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for 08f8d4ce92d3daa28bf960c05f8fc02f in 16ms, sequenceid=7, compaction requested=false 2023-03-27 03:55:54,487 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/namespace/08f8d4ce92d3daa28bf960c05f8fc02f/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-03-27 03:55:54,488 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f. 2023-03-27 03:55:54,488 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-03-27 03:55:54,488 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 08f8d4ce92d3daa28bf960c05f8fc02f: 2023-03-27 03:55:54,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1679889221357.08f8d4ce92d3daa28bf960c05f8fc02f. 2023-03-27 03:55:54,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 72b74f7fa0bbf3322278037ab0d1d7f8, disabling compactions & flushes 2023-03-27 03:55:54,489 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:54,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:54,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. after waiting 0 ms 2023-03-27 03:55:54,489 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:54,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/ns/testMinorCompaction/72b74f7fa0bbf3322278037ab0d1d7f8/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-27 03:55:54,495 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:54,496 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 72b74f7fa0bbf3322278037ab0d1d7f8: 2023-03-27 03:55:54,496 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1679889316507.72b74f7fa0bbf3322278037ab0d1d7f8. 2023-03-27 03:55:54,496 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ae7919e6ac686a9f3a2be09b01a0ff1a, disabling compactions & flushes 2023-03-27 03:55:54,496 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:54,496 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:54,496 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. after waiting 0 ms 2023-03-27 03:55:54,496 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:54,499 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/ae7919e6ac686a9f3a2be09b01a0ff1a/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-27 03:55:54,500 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:54,500 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ae7919e6ac686a9f3a2be09b01a0ff1a: 2023-03-27 03:55:54,500 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1679889338545.ae7919e6ac686a9f3a2be09b01a0ff1a. 2023-03-27 03:55:54,500 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f2819f01ee4f081befffef519bfab5a6, disabling compactions & flushes 2023-03-27 03:55:54,501 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6. 2023-03-27 03:55:54,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6. 2023-03-27 03:55:54,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6. after waiting 0 ms 2023-03-27 03:55:54,501 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6. 2023-03-27 03:55:54,506 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/f2819f01ee4f081befffef519bfab5a6/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-03-27 03:55:54,507 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6. 2023-03-27 03:55:54,508 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f2819f01ee4f081befffef519bfab5a6: 2023-03-27 03:55:54,508 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1679889251892.f2819f01ee4f081befffef519bfab5a6. 2023-03-27 03:55:54,508 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing faf5e912b9b1c732d5f1903dc1242b64, disabling compactions & flushes 2023-03-27 03:55:54,508 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:54,508 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:54,508 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. after waiting 0 ms 2023-03-27 03:55:54,508 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:54,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/faf5e912b9b1c732d5f1903dc1242b64/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-27 03:55:54,513 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:54,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for faf5e912b9b1c732d5f1903dc1242b64: 2023-03-27 03:55:54,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1679889293942.faf5e912b9b1c732d5f1903dc1242b64. 2023-03-27 03:55:54,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 647148480c2004d294d48558b731b404, disabling compactions & flushes 2023-03-27 03:55:54,513 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:55:54,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:55:54,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. after waiting 0 ms 2023-03-27 03:55:54,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:55:54,516 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/647148480c2004d294d48558b731b404/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-03-27 03:55:54,517 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:55:54,517 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 647148480c2004d294d48558b731b404: 2023-03-27 03:55:54,517 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1679889222235.647148480c2004d294d48558b731b404. 2023-03-27 03:55:54,517 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 08cac3032353cd1d586bb4d48f4a89b2, disabling compactions & flushes 2023-03-27 03:55:54,518 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:54,518 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:54,518 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. after waiting 0 ms 2023-03-27 03:55:54,518 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:54,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/08cac3032353cd1d586bb4d48f4a89b2/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-27 03:55:54,526 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:54,526 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 08cac3032353cd1d586bb4d48f4a89b2: 2023-03-27 03:55:54,526 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1679889293942.08cac3032353cd1d586bb4d48f4a89b2. 2023-03-27 03:55:54,526 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1002361bf1e6585024f86241020836f1, disabling compactions & flushes 2023-03-27 03:55:54,526 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1. 2023-03-27 03:55:54,526 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1. 2023-03-27 03:55:54,526 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1. after waiting 0 ms 2023-03-27 03:55:54,526 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1. 2023-03-27 03:55:54,528 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/1002361bf1e6585024f86241020836f1/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-27 03:55:54,530 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1. 2023-03-27 03:55:54,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1002361bf1e6585024f86241020836f1: 2023-03-27 03:55:54,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1679889251892.1002361bf1e6585024f86241020836f1. 2023-03-27 03:55:54,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c0982fa95f3b2e90e5d67d21d9f32707, disabling compactions & flushes 2023-03-27 03:55:54,530 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707. 2023-03-27 03:55:54,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707. 2023-03-27 03:55:54,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707. after waiting 0 ms 2023-03-27 03:55:54,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707. 2023-03-27 03:55:54,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testScannerOnBulkLoadRefHFiles/c0982fa95f3b2e90e5d67d21d9f32707/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-03-27 03:55:54,534 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707. 2023-03-27 03:55:54,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c0982fa95f3b2e90e5d67d21d9f32707: 2023-03-27 03:55:54,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1679889251892.c0982fa95f3b2e90e5d67d21d9f32707. 2023-03-27 03:55:54,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1b540ab2fb7dd087d0e66746ccea8753, disabling compactions & flushes 2023-03-27 03:55:54,534 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:55:54,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:55:54,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. after waiting 0 ms 2023-03-27 03:55:54,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:55:54,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/1b540ab2fb7dd087d0e66746ccea8753/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-03-27 03:55:54,538 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:55:54,538 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1b540ab2fb7dd087d0e66746ccea8753: 2023-03-27 03:55:54,538 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1679889278118.1b540ab2fb7dd087d0e66746ccea8753. 2023-03-27 03:55:54,538 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a7921456431e437526c5b87521696821, disabling compactions & flushes 2023-03-27 03:55:54,538 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:55:54,538 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:55:54,538 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. after waiting 0 ms 2023-03-27 03:55:54,538 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:55:54,541 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionWithWeeklyPolicy/a7921456431e437526c5b87521696821/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-03-27 03:55:54,541 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:55:54,542 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a7921456431e437526c5b87521696821: 2023-03-27 03:55:54,542 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1679889255948.a7921456431e437526c5b87521696821. 2023-03-27 03:55:54,542 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d0bf0165da69488ffc28ab9679dbd8e9, disabling compactions & flushes 2023-03-27 03:55:54,542 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:54,542 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:54,542 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. after waiting 0 ms 2023-03-27 03:55:54,542 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:54,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMajorCompactionFromAdmin/d0bf0165da69488ffc28ab9679dbd8e9/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-03-27 03:55:54,545 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:54,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d0bf0165da69488ffc28ab9679dbd8e9: 2023-03-27 03:55:54,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1679889338545.d0bf0165da69488ffc28ab9679dbd8e9. 2023-03-27 03:55:54,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a96b2587b7874c72fc83ce1239d003a2, disabling compactions & flushes 2023-03-27 03:55:54,545 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:54,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:54,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. after waiting 0 ms 2023-03-27 03:55:54,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:54,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testCompactionWithHFileLink/a96b2587b7874c72fc83ce1239d003a2/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-03-27 03:55:54,549 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:54,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a96b2587b7874c72fc83ce1239d003a2: 2023-03-27 03:55:54,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1679889293942.a96b2587b7874c72fc83ce1239d003a2. 2023-03-27 03:55:54,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 5f700223782e72e7df2f686db6ddd489, disabling compactions & flushes 2023-03-27 03:55:54,549 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:54,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:54,550 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. after waiting 0 ms 2023-03-27 03:55:54,550 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:54,553 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/default/testMinorCompactionWithMonthlyPolicy/5f700223782e72e7df2f686db6ddd489/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-03-27 03:55:54,553 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:54,553 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 5f700223782e72e7df2f686db6ddd489: 2023-03-27 03:55:54,553 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1679889329720.5f700223782e72e7df2f686db6ddd489. 2023-03-27 03:55:54,626 DEBUG [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-27 03:55:54,827 DEBUG [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-27 03:55:54,841 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/.tmp/info/5b55c1a75dff498ab0a3c0cb83659939 2023-03-27 03:55:54,846 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5b55c1a75dff498ab0a3c0cb83659939 2023-03-27 03:55:55,027 DEBUG [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-27 03:55:55,227 DEBUG [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-03-27 03:55:55,252 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/.tmp/rep_barrier/0a9af90255464ac1a2bbd478e1eef8e4 2023-03-27 03:55:55,257 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a9af90255464ac1a2bbd478e1eef8e4 2023-03-27 03:55:55,264 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/.tmp/table/812a939ff9e349d8a6208a8406138c89 2023-03-27 03:55:55,267 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 812a939ff9e349d8a6208a8406138c89 2023-03-27 03:55:55,268 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/.tmp/info/5b55c1a75dff498ab0a3c0cb83659939 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/info/5b55c1a75dff498ab0a3c0cb83659939 2023-03-27 03:55:55,271 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5b55c1a75dff498ab0a3c0cb83659939 2023-03-27 03:55:55,271 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/info/5b55c1a75dff498ab0a3c0cb83659939, entries=226, sequenceid=130, filesize=37.9 K 2023-03-27 03:55:55,272 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/.tmp/rep_barrier/0a9af90255464ac1a2bbd478e1eef8e4 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/rep_barrier/0a9af90255464ac1a2bbd478e1eef8e4 2023-03-27 03:55:55,275 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0a9af90255464ac1a2bbd478e1eef8e4 2023-03-27 03:55:55,275 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/rep_barrier/0a9af90255464ac1a2bbd478e1eef8e4, entries=1, sequenceid=130, filesize=5.0 K 2023-03-27 03:55:55,276 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/.tmp/table/812a939ff9e349d8a6208a8406138c89 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/table/812a939ff9e349d8a6208a8406138c89 2023-03-27 03:55:55,278 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 812a939ff9e349d8a6208a8406138c89 2023-03-27 03:55:55,279 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/table/812a939ff9e349d8a6208a8406138c89, entries=27, sequenceid=130, filesize=6.6 K 2023-03-27 03:55:55,281 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 855ms, sequenceid=130, compaction requested=false 2023-03-27 03:55:55,285 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-03-27 03:55:55,285 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-03-27 03:55:55,285 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-03-27 03:55:55,285 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-03-27 03:55:55,285 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-03-27 03:55:55,297 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-03-27 03:55:55,298 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-03-27 03:55:55,428 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,39727,1679889218560; all regions closed. 2023-03-27 03:55:55,436 DEBUG [RS:0;jenkins-hbase4:39727] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/oldWALs 2023-03-27 03:55:55,436 INFO [RS:0;jenkins-hbase4:39727] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39727%2C1679889218560.meta:.meta(num 1679889221116) 2023-03-27 03:55:55,441 DEBUG [RS:0;jenkins-hbase4:39727] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/oldWALs 2023-03-27 03:55:55,441 INFO [RS:0;jenkins-hbase4:39727] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39727%2C1679889218560:(num 1679889220449) 2023-03-27 03:55:55,441 DEBUG [RS:0;jenkins-hbase4:39727] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:55:55,441 INFO [RS:0;jenkins-hbase4:39727] regionserver.LeaseManager(133): Closed leases 2023-03-27 03:55:55,441 INFO [RS:0;jenkins-hbase4:39727] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-03-27 03:55:55,442 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-27 03:55:55,443 INFO [RS:0;jenkins-hbase4:39727] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39727 2023-03-27 03:55:55,453 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-27 03:55:55,453 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,39727,1679889218560 2023-03-27 03:55:55,453 ERROR [Listener at localhost/36131-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@247c91ab rejected from java.util.concurrent.ThreadPoolExecutor@204311fc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 531] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-27 03:55:55,453 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-03-27 03:55:55,453 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,39727,1679889218560] 2023-03-27 03:55:55,453 ERROR [Listener at localhost/36131-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@145fe2ae rejected from java.util.concurrent.ThreadPoolExecutor@204311fc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 531] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-27 03:55:55,454 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,39727,1679889218560; numProcessing=1 2023-03-27 03:55:55,457 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,39727,1679889218560 already deleted, retry=false 2023-03-27 03:55:55,457 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,39727,1679889218560 expired; onlineServers=0 2023-03-27 03:55:55,457 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,38271,1679889217353' ***** 2023-03-27 03:55:55,457 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-03-27 03:55:55,458 DEBUG [M:0;jenkins-hbase4:38271] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6831802d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-03-27 03:55:55,458 INFO [M:0;jenkins-hbase4:38271] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,38271,1679889217353 2023-03-27 03:55:55,458 INFO [M:0;jenkins-hbase4:38271] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,38271,1679889217353; all regions closed. 2023-03-27 03:55:55,458 DEBUG [M:0;jenkins-hbase4:38271] ipc.AbstractRpcClient(494): Stopping rpc client 2023-03-27 03:55:55,458 DEBUG [M:0;jenkins-hbase4:38271] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-03-27 03:55:55,458 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-03-27 03:55:55,458 DEBUG [M:0;jenkins-hbase4:38271] cleaner.HFileCleaner(317): Stopping file delete threads 2023-03-27 03:55:55,458 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1679889220119] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1679889220119,5,FailOnTimeoutGroup] 2023-03-27 03:55:55,458 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1679889220125] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1679889220125,5,FailOnTimeoutGroup] 2023-03-27 03:55:55,458 INFO [M:0;jenkins-hbase4:38271] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-03-27 03:55:55,458 INFO [M:0;jenkins-hbase4:38271] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-03-27 03:55:55,459 INFO [M:0;jenkins-hbase4:38271] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-03-27 03:55:55,459 DEBUG [M:0;jenkins-hbase4:38271] master.HMaster(1512): Stopping service threads 2023-03-27 03:55:55,459 INFO [M:0;jenkins-hbase4:38271] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-03-27 03:55:55,460 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-03-27 03:55:55,460 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-03-27 03:55:55,460 ERROR [M:0;jenkins-hbase4:38271] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-19,5,PEWorkerGroup] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-03-27 03:55:55,460 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-03-27 03:55:55,461 INFO [M:0;jenkins-hbase4:38271] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-03-27 03:55:55,461 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-03-27 03:55:55,461 DEBUG [M:0;jenkins-hbase4:38271] zookeeper.ZKUtil(398): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-03-27 03:55:55,461 WARN [M:0;jenkins-hbase4:38271] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-03-27 03:55:55,461 INFO [M:0;jenkins-hbase4:38271] assignment.AssignmentManager(315): Stopping assignment manager 2023-03-27 03:55:55,462 INFO [M:0;jenkins-hbase4:38271] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-03-27 03:55:55,462 DEBUG [M:0;jenkins-hbase4:38271] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-03-27 03:55:55,462 INFO [M:0;jenkins-hbase4:38271] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-27 03:55:55,462 DEBUG [M:0;jenkins-hbase4:38271] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-27 03:55:55,462 DEBUG [M:0;jenkins-hbase4:38271] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-03-27 03:55:55,462 DEBUG [M:0;jenkins-hbase4:38271] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-27 03:55:55,462 INFO [M:0;jenkins-hbase4:38271] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.67 KB heapSize=598.82 KB 2023-03-27 03:55:55,557 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-27 03:55:55,557 INFO [RS:0;jenkins-hbase4:39727] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,39727,1679889218560; zookeeper connection closed. 2023-03-27 03:55:55,557 ERROR [Listener at localhost/36131-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6c9affc7 rejected from java.util.concurrent.ThreadPoolExecutor@204311fc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 531] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-27 03:55:55,557 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): regionserver:39727-0x101241b1daf0001, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-27 03:55:55,557 ERROR [Listener at localhost/36131-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6ecfac4e rejected from java.util.concurrent.ThreadPoolExecutor@204311fc[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 531] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-27 03:55:55,558 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@72fdf3a6] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@72fdf3a6 2023-03-27 03:55:55,558 INFO [Listener at localhost/36131] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-03-27 03:55:55,874 INFO [M:0;jenkins-hbase4:38271] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.67 KB at sequenceid=1011 (bloomFilter=true), to=hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/1c17d6dfa4194663a82fecc380ca3b33 2023-03-27 03:55:55,877 INFO [M:0;jenkins-hbase4:38271] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1c17d6dfa4194663a82fecc380ca3b33 2023-03-27 03:55:55,878 DEBUG [M:0;jenkins-hbase4:38271] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/1c17d6dfa4194663a82fecc380ca3b33 as hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/1c17d6dfa4194663a82fecc380ca3b33 2023-03-27 03:55:55,881 INFO [M:0;jenkins-hbase4:38271] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1c17d6dfa4194663a82fecc380ca3b33 2023-03-27 03:55:55,881 INFO [M:0;jenkins-hbase4:38271] regionserver.HStore(1080): Added hdfs://localhost:43651/user/jenkins/test-data/ad3d395a-9c45-9dc1-7edb-84cc3b862001/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/1c17d6dfa4194663a82fecc380ca3b33, entries=123, sequenceid=1011, filesize=33.3 K 2023-03-27 03:55:55,882 INFO [M:0;jenkins-hbase4:38271] regionserver.HRegion(2948): Finished flush of dataSize ~512.67 KB/524977, heapSize ~598.80 KB/613176, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 420ms, sequenceid=1011, compaction requested=false 2023-03-27 03:55:55,885 INFO [M:0;jenkins-hbase4:38271] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-03-27 03:55:55,885 DEBUG [M:0;jenkins-hbase4:38271] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-03-27 03:55:55,888 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-03-27 03:55:55,888 INFO [M:0;jenkins-hbase4:38271] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-03-27 03:55:55,889 INFO [M:0;jenkins-hbase4:38271] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38271 2023-03-27 03:55:55,892 DEBUG [M:0;jenkins-hbase4:38271] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,38271,1679889217353 already deleted, retry=false 2023-03-27 03:55:55,994 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-27 03:55:55,994 INFO [M:0;jenkins-hbase4:38271] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,38271,1679889217353; zookeeper connection closed. 2023-03-27 03:55:55,994 ERROR [Listener at localhost/36131-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7dcfa251 rejected from java.util.concurrent.ThreadPoolExecutor@2aebe15a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 906] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-27 03:55:55,994 DEBUG [Listener at localhost/36131-EventThread] zookeeper.ZKWatcher(600): master:38271-0x101241b1daf0000, quorum=127.0.0.1:62246, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-03-27 03:55:55,994 ERROR [Listener at localhost/36131-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4426cf36 rejected from java.util.concurrent.ThreadPoolExecutor@2aebe15a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 906] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-03-27 03:55:55,996 WARN [Listener at localhost/36131] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-03-27 03:55:56,000 INFO [Listener at localhost/36131] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-27 03:55:56,009 WARN [BP-154749535-172.31.14.131-1679889214022 heartbeating to localhost/127.0.0.1:43651] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-03-27 03:55:56,009 WARN [BP-154749535-172.31.14.131-1679889214022 heartbeating to localhost/127.0.0.1:43651] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-154749535-172.31.14.131-1679889214022 (Datanode Uuid 934c2615-be1e-4545-af16-f1d75719e819) service to localhost/127.0.0.1:43651 2023-03-27 03:55:56,012 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/cluster_3244e29f-4978-4f89-598f-7715da9c7d7c/dfs/data/data1/current/BP-154749535-172.31.14.131-1679889214022] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-27 03:55:56,012 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/65ab68d5-2bf4-0664-903b-f515136538a7/cluster_3244e29f-4978-4f89-598f-7715da9c7d7c/dfs/data/data2/current/BP-154749535-172.31.14.131-1679889214022] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-03-27 03:55:56,044 INFO [Listener at localhost/36131] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-03-27 03:55:56,088 INFO [Listener at localhost/36131] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-03-27 03:55:56,191 INFO [Listener at localhost/36131] hbase.HBaseTestingUtility(1293): Minicluster is down